Blog
Getting Started With the Milvus JavaScript Client

Getting Started With the Milvus JavaScript Client

Jul 28, 20239 min read

Milvus is a robust open-source database for vector search. Its ability to scale horizontally for nearly any load makes it the best choice for machine learning deployments. It also features best-of-class tooling, including for JavaScript clients. In this post, we'll show you how to get started with Milvus and JavaScript.

The Milvus node.js SDK(Client) allows you to build flexible and powerful data handling capabilities right into your JavaScript environment. It uses Node.js to connect to a Milvus server, providing you with complete access to all database operations. You can use the client to create collections, insert data, create indexes, perform queries and searches, and more.

We'll start by setting up the environment required to run the code, including a local server instance. Once we're up and running, you'll move on to connecting to the Milvus database. This foundational step ensures you're plugged into the vast resources Milvus offers.

Then you'll create a collection in Milvus. With your collection set up, we'll finally get into the heart of data operations: inserting and querying data.

Let's get started.

Prerequisites

Before you start, you'll need a few things installed on your development system.

We'll use Milvus Lite for this tutorial, so you'll need Python 3.8 or greater. You'll also need Node.js version 12 or greater, npm version 12 or greater, or yarn, to install packages.

After you prepare everything, let’s install and run Milvus Lite and JavaScript Client.

Install and run Milvus Lite

A best practice for running Python applications like Milvus Lite is to use a virtual environment. Create an environment, source it, and upgrade pip.

[egoebelbecker@ares examples]$ python3 -m venv ./venv
[egoebelbecker@ares examples]$ source venv/bin/activate
(venv) [egoebelbecker@ares examples]$ pip install --upgrade pip
Requirement already satisfied: pip in ./venv/lib64/python3.11/site-packages (22.3.1)
Collecting pip
Using cached pip-23.2-py3-none-any.whl (2.1 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 22.3.1
Uninstalling pip-22.3.1:
Successfully uninstalled pip-22.3.1
Successfully installed pip-23.2

Next, install the milvus library from PyPi.

(venv) [egoebelbecker@ares examples]$ pip install milvus
Collecting milvus
  Obtaining dependency information for milvus from https://files.pythonhosted.org/packages/c1/a6/599ba1f8615e6ec73208ab614f5daf7ab27097070fbbd1715a1b5cc5e578/milvus-2.2.11-py3-none-manylinux2014_x86_64.whl.metadata
  Downloading milvus-2.2.11-py3-none-manylinux2014_x86_64.whl.metadata (6.7 kB)
Using cached milvus-2.2.11-py3-none-manylinux2014_x86_64.whl (51.7 MB)
Installing collected packages: milvus
Successfully installed milvus-2.2.11
(venv) [egoebelbecker@ares examples]$

Finally, run your server.

(venv) [egoebelbecker@ares examples]$ milvus-server 

    __  _________ _   ____  ______
   /  |/  /  _/ /| | / / / / / __/
  / /|_/ // // /_| |/ / /_/ /\ \
 /_/  /_/___/____/___/\____/___/ {Lite}

 Welcome to use Milvus!

 Version:   v2.2.11-lite
 Process:   505236
 Started:   2023-07-20 14:18:37
 Config:    /home/egoebelbecker/.milvus.io/milvus-server/2.2.11/configs/milvus.yaml
 Logs:      /home/egoebelbecker/.milvus.io/milvus-server/2.2.11/logs

 Ctrl+C to exit ...

Install the Milvus node.js SDK(Client)

First, initialize npm in your working directory.

[egoebelbecker@ares examples]$ npm init
[egoebelbecker@ares examples]$ npm install @zilliz/milvus2-sdk-node

Now, you're ready to write some code!

Getting started with Milvus and JavaScript

After you installed Milvus and JavaScript, you’re ready to run Milvus and JavaScript.

Connect to Milvus

You can connect to and communicate with Milvus via the MilvusClient class.

Milvus offers several options to connect, such as username, password, and server address. Since you're using Milvus Lite, you only need to tell it where to connect.

Create a MilvusClient and check the connectStatus field to ensure it's connected:

import { MilvusClient } from '@zilliz/milvus2-sdk-node';

const milvusClient = new MilvusClient({
    address: 'localhost:19530',
});

console.log("Connection status: " + milvusClient.connectStatus)

Here's the output:

Connection status: 1

Create a database

Your server is empty, so let's start by creating a database.

Most Milvus Javascript API calls involve a network request, so they return a promise. They're also not guaranteed to execute in the order you list them in your code, so you must be mindful of timing and using tools like promise chaining, code blocks, and await to ensure functions execute in the order you expect.

To make the code easier to read, I'll use individual code blocks and print the promise when it completes. I'll also use await when necessary as a "brute force" approach to ensuring code executes in the correct order.

For production code, you'd probably use a different approach.

This code creates a database with createDatabase, using await to keep the code from proceeding until it is complete. Then it verifies that the new database is thereby calling listDatabases.

CreateDatabase only needs a database name.

import { MilvusClient } from '@zilliz/milvus2-sdk-node';

(async () => {
  const milvusClient = new MilvusClient({
    address: 'localhost:19530',
  });
  console.log("Connection status: " + milvusClient.connectStatus)

  const createDb = await milvusClient.createDatabase({ db_name: 'my_db' });
  console.log('Database is created', createDb);

  const listDatabases = await milvusClient.listDatabases();
  console.log('list databases', listDatabases);
})();

The results are as follows:

Connection status: 1
Database is created { error_code: 'Success', reason: '' }
list databases {
  db_names: [ 'my_db', 'default' ],
  status: { error_code: 'Success', reason: '' }
}

Create a collection

Next, it's time to create a collection so that you can add some data to it.

Before you create or use a collection, you need to use the database that contains it. So, after connecting, this script uses my_db.

Then, it creates the new collection with createCollection and checks it with describeCollection. A new collection needs a schema with data types, so you must import DataType into the script.

import { MilvusClient, DataType } from '@zilliz/milvus2-sdk-node';

(async () => {
  const milvusClient = new MilvusClient({
    address: 'localhost:19530',
  });
  console.log('Connection status: ' + milvusClient.connectStatus);

  await milvusClient.use({ db_name: 'my_db' });

  const create = await milvusClient.createCollection({
    collection_name: 'sample_collection',
    fields: [
      {
        name: 'age',
        description: 'ID field',
        data_type: DataType.Int64,
        is_primary_key: true,
        autoID: true,
      },
      {
        name: 'vector',
        description: 'Vector field',
        data_type: DataType.FloatVector,
        dim: 8,
      },
      { name: 'height', description: 'int64 field', data_type: DataType.Int64 },
      {
        name: 'name',
        description: 'VarChar field',
        data_type: DataType.VarChar,
        max_length: 128,
      },
    ],
  });
  console.log('Create collection is finished.', create);

  await milvusClient.describeCollection({ collection_name: 'sample_collection'});

  console.log('describe collection', describeCollection);
 })();

The script succeeds with information about the new collection:

Connection status: 1
new Database is using { error_code: 'Success', reason: '' }
Create collection is finished. { error_code: 'Success', reason: '' }
describe collection {
  virtual_channel_names: [ 'by-dev-rootcoord-dml_1_443014985196255045v0' ],
  physical_channel_names: [ 'by-dev-rootcoord-dml_1' ],
  aliases: [],
  start_positions: [],
  properties: [],
  status: { error_code: 'Success', reason: '' },
  schema: {
    fields: [ [Object], [Object], [Object], [Object] ],
    name: 'sample_collection',
    description: '',
    autoID: false,
    enable_dynamic_field: false
  },
  collectionID: '443014985196255045',
  created_timestamp: '443015720996700166',
  created_utc_timestamp: '1689970859515',
  shards_num: 1,
  consistency_level: 'Bounded',
  collection_name: 'sample_collection',
  db_name: 'my_db',
  num_partitions: '1'
}

Add and retrieve data

Now it's time to add some data to the collection.

First, we need the data. Usually, your code reads or assembles data from somewhere, but you'll use some hard-coded values.

After declaring an array of values, this script uses an InsertReq to insert it by name into the new collection:

import { MilvusClient, InsertReq } from '@zilliz/milvus2-sdk-node';

(async () => {
  const milvusClient = new MilvusClient({
    address: 'localhost:19530',
  });
  console.log('Connection status: ' + milvusClient.connectStatus);

  const useDb = await milvusClient.use({ db_name: 'my_db' });
  console.log('new Database is using', useDb);
  
  const vectorsData = [
    {
      vector: [
        0.11878310581111173, 0.9694947902934701, 0.16443679307243175,
        0.5484226189097237, 0.9839246709011924, 0.5178387104937776,
        0.8716926129208069, 0.5616972243831446,
      ],
      height: 20405,
      name: 'zlnmh',
    },
    {
      vector: [
        0.9992090731236536, 0.8248790611809487, 0.8660083940881405,
        0.09946359318481224, 0.6790698063908669, 0.5013786801063624,
        0.795311915725105, 0.9183033261617566,
      ],
      height: 93773,
      name: '5lr9y',
    },
    {
      vector: [
        0.8761291569818763, 0.07127366044153227, 0.775648976160332,
        0.5619757601304878, 0.6076543120476996, 0.8373907516027586,
        0.8556140171597648, 0.4043893119391049,
      ],
      height: 85122,
      name: 'nes0j',
    },
    {
      vector: [
        0.5849602436079879, 0.5108258101682586, 0.8250884731578105,
        0.7996354835509332, 0.8207766774911736, 0.38133662902290566,
        0.7576720055508186, 0.4393152967662368,
      ],
      height: 92037,
      name: 'ct2li',
    },
    {
      vector: [
        0.3768133716738886, 0.3823259261020866, 0.7906232829855262,
        0.31693696726284193, 0.3731715403499176, 0.3300751870649885,
        0.22353556137796238, 0.38062799545615444,
      ],
      height: 31400,
      name: '6ghrg',
    },
    {
      vector: [
        0.0007531778212483964, 0.12941566118774994, 0.9340164428788116,
        0.3795768837758642, 0.4532443258064389, 0.596455163143,
        0.9529469158782906, 0.7692465408044873,
      ],
      height: 1778,
      name: 'sb7mt',
    },
  ];

  const params: InsertReq = {
    collection_name: 'sample_collection',
    fields_data: vectorsData,
  };
  // insert data into collection
  const insert = await milvusClient.insert(params);
  console.log('Inserted data:', insert);
})();

The API returns information about what you inserted:

Connection status: 1
new Database is using { error_code: 'Success', reason: '' }
Inserted data: {
  succ_index: [ 0, 1, 2, 3, 4, 5 ],
  err_index: [],
  status: { error_code: 'Success', reason: '' },
  IDs: { int_id: { data: [Array] }, id_field: 'int_id' },
  acknowledged: false,
  insert_cnt: '6',
  delete_cnt: '0',
  upsert_cnt: '0',
  timestamp: '443015922585698312'
}

Now, you have some data to query. But before you started running queries in production, you'd probably add an index to your collection. This adds an index on the vector field:

const createIndex = await milvusClient.createIndex({
   collection_name: 'sample_collection',
   field_name: 'vector',
   metric_type: 'L2',
});

To run this code, your script still needs to connect and use the associated database.

Finally, you can run a query written in any expression or query supported by Milvus. For this one, it simply filters items based on age.

import { MilvusClient } from '@zilliz/milvus2-sdk-node';

(async () => {
  const milvusClient = new MilvusClient({
    address: 'localhost:19530',
  });

  const useDb = await milvusClient.use({ db_name: 'my_db' });
  console.log('new Database is using', useDb);
  
  const load = await milvusClient.loadCollectionSync({ collection_name: 'sample_collection' });

  const query = await milvusClient.query({
    collection_name: 'sample_collection',
    filter: 'age > 0',
    output_fields: ['age', 'vector'],
    limit: 100,
  });
  console.log('query result', query);
})();

Here's the result:

new Database is using { error_code: 'Success', reason: '' }
Query time: 235.806ms
query result {
  status: { error_code: 'Success', reason: '' },
  data: [
    { vector: [Array], age: '443014985196258910' },
    { vector: [Array], age: '443014985196258911' },
    { vector: [Array], age: '443014985196258912' },
    { vector: [Array], age: '443014985196258913' },
    { vector: [Array], age: '443014985196258914' },
    { vector: [Array], age: '443014985196258915' }
  ]
}

Search vectors

Once your collection is loaded into memory, you can perform vector similarity search on it.

import { MilvusClient } from '@zilliz/milvus2-sdk-node';

(async () => {
  const milvusClient = new MilvusClient({
    address: 'localhost:19530',
  });

  const useDb = await milvusClient.use({ db_name: 'my_db' });
  console.log('new Database is using', useDb);
  
  const load = await milvusClient.loadCollectionSync({ collection_name: 'sample_collection' });

  const query = await milvusClient.search({
    collection_name: 'sample_collection',
    vector: [1,2,3,4,5,6,7,8],
    filter: 'age > 0',
    output_fields: ['id', 'age'],
    limit: 5,
  });
  console.log('query result', query);
})();

Here's the result:

  Search result {
  status: { error_code: 'Success', reason: '' },
  results: [
    { score: 0, id: '442936641561652921', age: '442936641561652921' },
    {
      score: 1.0004949569702148,
      id: '442936641561652924',
      age: '442936641561652924'
    },
    {
      score: 1.6779069900512695,
      id: '442936641561652926',
      age: '442936641561652926'
    },
    {
      score: 1.7160398960113525,
      id: '442936641561652922',
      age: '442936641561652922'
    },
    {
      score: 1.718210220336914,
      id: '442936641561652925',
      age: '442936641561652925'
    }
  ]
}

Summary

We've set up a development environment for working with Milvus and the Milvus JavaScript. We connected to a server, created a database and a collection, inserted data and retrieved it, and performed similarity vector searches. As you've seen, working with vector data in JavaScript with Milvus couldn't be simpler!

Add the power of vector search to your web apps today! While you're at it, take a look at a Zilliz Cloud free trial and see how you can have a cloud-ready, reliable, and high-performance Milvus cluster running in minutes.

About the author

This post was written by Eric Goebelbecker. Eric has worked in the financial markets in New York City for 25 years, developing infrastructure for market data and financial information exchange (FIX) protocol networks. He loves to talk about what makes teams effective (or not so effective!).

Eric Goebelbecker
Eric Goebelbecker has worked in the financial markets in New York City for 25 years, developing infrastructure for market data and financial information exchange (FIX) protocol networks. He loves to talk about what makes teams effective (or not so effective!).