- Events
Unlocking the power of vector search in Zilliz Cloud
Webinar
Unlocking the power of vector search in Zilliz Cloud
Join the Webinar
Loading...
What will you learn?
Zilliz is excited to announce the general availability of Zilliz Cloud, a fully-managed Milvus service made by the creators of Milvus. This new multi-cloud service (AWS and GCP) offers consistent management, security, and governance experience across all clouds. It provides reliability & resiliency built-in, 99.99% uptime SLA, and zero data corruption. Frank Liu, Director of Operations & ML Architect at Zilliz, will demonstrate key product features in this webinar.
Topics covered:
- New cluster sizes to support over a billion multi-dimensional vectors
- Embedding model integrations (OpenAI, Hugging Face, Cohere)
- Powerful Backup and restore
- Simple Migration from Milvus to Zilliz
- All the latest Milvus features
- Performance improvements and more!
Accession unlocking the power of vector Search Whistle as cloud and today's speaker my colleague Frank Liu. Frank is the director operations and a machine learning architect here at zillows prior to zilla's Frank co-founded Orion Innovations and ml-powered indoor positioning startup based in Shanghai and worked as an ml engineer at Yahoo in San Francisco. Frank is very smart and holds an MS nvs in electrical engineering from Stanford. Welcome FrankLook everybody and in this session today, I'll be talking about zilla's Cloud Vector search in general and really how you can get started with sales Cloud. What's in the features are and what's in the use cases are as well.
Right? So let's dive right into it. Yeah quick introduction. My name is Frank director of operations. Ml architect here at zilla's. There's my socialism down there if you want to get in touch with me.
And first thing I want to do is talk a little bit about zoo is who we are and you know, where we come from. Right? So Zeus Cloud itself. It is a vector database but it's more the fully managed version of notes. You know, this is really the world's most popular open source Factor. This might have heard it and what it really does is it gives you a very powerful flexible way of storing searching and indexing a variety of embeddings and metadata from these machine learning models, whatever whatever other unstructure that you have.
We have really Lightning Fast queries on any data set size cost effective storage of vectors will get to that a little more in a later session and absolutely zero officer ahead. So it is a fully mentioned service go online click click a couple of buttons and you'll be able to spin up production ready for database and just a manner of seconds. Right? So it's really something that is meant to be full service. It is the no hassle required. So yeah, quick quick sort of preview of what's coming up in this session.
First is I'll give an introductions into those Cloud. I'll talk about some of the features and functionality of those clouds follow on onto that and I know a lot of folks who are watching this this session are probably using those already the community edition of so as Cloud, I'll talk a bit about migrating from those as well. I'll follow there with just a couple of use cases and mind you those will be some use cases some popular ones that we've seen recently and throughout the course that we will just Cloud development and then I'll go through a quick demo more so quick tour I suppose rather than a quick demo. So without further Ado, let's get into it. Right so a quick introduction to zilla's Cloud.
What is those Cloud now? I touched upon this a little bit already but it is a platform. It is a production ready Vector database that's built on top of milvis and optimized for whatever your application needs of your application requires very very large application require. Lots of storage. We have a way to do that. If you need very very high performance High QPS High throughput, you know, we have ways of doing that as well highly elastic and scalable.
So you can always scale up scale down your close to your instance as you see fits pay as you go right. So there's no sort of you know, as you use the vector database as you decide to stop it depending on how much storage you you're using depending on the vectors your story inside of it your usage in terms of the compute. That is something we won't. This was right where to put it. We won't extend that we wanted we won't enforce any strict.
Let's say one month six month one of your timelines and we're multi-cloud as well. Right? So right now we're on AWS and gcp would definitely with more coming on the way and we have 99. 99% uptime SLA would absolutely zero data Crush. So if you do store your vectors in those Cloud you can be absolutely sure thatThey will be there when you need it. You're never going to be able to you're gonna have you're never gonna have lost data.
We have a lot of safeguards built to built into place to prevent a lot of that from happening. Enterprise security and governance, right? So there'sI think a lot to be said for creating the expected databases these vectors search engines, but without having a lot of those Enterprise features that you need in them. And those Cloud actually is great A in terms of any, you know, this sense right here and we're also integrated with a lot of key embedding models. So if you want to use large language models from Lane chain, if you want to use open source models from huggling face, if you want to use the opening and betting APR or the code here embedding API or Sports Vision models torch. Are you models we have ways of doing that those Cloud provides very very easy integration a lot of those models.
We have great examples on the website as well. And we have support sdks Python and Java. So depending on your use case depending on your needs you're able to use that as fit, okay. So now we've gotten that out of the way. I do want to give folks a bit of a refresher on you know this and what it's used for how it's typically used in a production environment and that will give us a veryI suppose smooth transition into talking more about this card as well.
So milvis is as I mentioned earlier. It's the world's most popular open source Vector database and as a vector in database, it's really meant to store a variety of different types of unstructured data like as we like to call it. So give a variety of let's say documents. You variety of text and using embedding models and using your own models. You will actually turn these into what are called these embedding vectors and these vectors are a really good and a really powerful representation of your Innovative data.
So if I have let's say document, so let's say a page long or five paragraphs. I have things that I want to index. I can turn those into vectors that represents semantically that input data very very well, right and then we store those dog. We store those vectors inside of no lists. No this provides the storage indexing and search capabilities that you want.
It does. They're very very scalable and Brilliant scale away billion. With billion scale vectors, excuse me. So let's talk about movies. Right and this this is sort of lead into some of the some things that that was over a little later in those Cloud.
Now, these are some of the applications absolutely not a complete list as semantic text search there's a lot of work that's being done recently integrating back your databases with large line with models like a similarities search another one of my favorite ones. They really turn these very very complex molecular structures since you fixed length vectors the search really do AI for Discovery applications such as that using a vector database and also some reverse image search as well. So if I want to be able to search for an image using other images, that's one of the ways to do it. And again, this is just a sample of a variety of different applications that you can build with those with effective database, right? Well, I do want to spend a bit of extra time. About Vector databases with llms and a key.
Way that we see a lot of bill. There's a lot of Engineers a lot of developers using sales cloudloms isIf you have a corpus of documents, or if you have let's say you're in the air company you're in finance industry. You have internal financial documents and you want to be able to get your autoregressive language models to answer the questions and with summarize those documents a very key very common way that we see that being done is using the vector database to store all of your documents and to query it. If you let's say give your lungs multiple prompt use that promise query in the factory database, you'll be retrieve all development documents related to that query relates to that prompts inject that as extra problems into your large line with model and you can get it to answer questions about that about your query. You can really inject more domain knowledge into your into your application.
And that's one of the very very powerful ways that we've seen Vector databases such as those Cloud being used in production today. Good. I wanted to take an extra minute or two to really highlight that. So why choose zilla's over Movis? And this is more, you know, this is more comparison of what Zeals provides versus what Novus provides and again, both are fantastic databases Zeal is this cloud is build on top of novices. And as you can see here, they both provide really all the functionality that you would want in terms of being able to terms of the underlying better search capability.
They both provide very scalable efficient easy to use Vector similarity search and you can find some a lot of your different parameters to be able to get exactly the contender performance you need for your application else will provide hybrid search as well. So you can basically the ability to do both Vector as well as metadata search as well if they're doing if I have vectors and I have a scalar Fields associated with them. I can search over I can create indexes over all those fields of all of that. Support for a variety of different types of schemas to mentioned here float and binary vectors by the same time. We also have support for Strings as well.
Boolean values integers. So on and so forth so you can really Define the scheme to be exactly what you need. And we also have the capability to do automatic indexing. You have excuse me, Zeus cloud has capable to do automatic indexing Novus. You can in a way you can also do automatic indexing as well, but it doesn't require a bit of an extra code of bit of extra logic to do.
So on those types of specify other index that a little bit more later and you'll be able to really just that will be able to do the exact type next thing they want and we also have a variety of compatible sdks most Community Edition as well for Zeus. We have Python and Java. These are really fully supported 100% tested to make sure that they really work with this flower, right?And I also want to get into some of the features that those public Bots that novice doesn't the first is I think obviously this looks positively managed. So you have zero Ops overhead. Entirely automated a lot of features that you want automatic upgrades backup store.
These are implemented in zillow's Cloud. You don't have to worry about that and we have very very high. We have high availability as well and four nine up time SLA with failover so that if anything ever happens to your vector database and always look forward that data. Blessed scaling I alluded to this a little bit earlier. If you want to be able to do billion scale regular search.
If you want to be able to do reset a very very rapid thousand 10,000 queries per second. You want to be able to scale up your cluster? I think we should we have also a variety of machines types that are optimized to perform in a to perform the best free application. And again again, who is a little bit later as well. We have as much storage as you want. So if you store let's say 100 million vectors, if you store billion vectors historic writing mending along with those there's great ways, you know can support that to however much you need and one of my favorite features as well is the amazing user friendly UI you can use to manage to view to monitor to look at your Midwest instance really at any time management.
Dimensions there, and we also have resource monitors as well. Right? So if you want to be able to set a limit sending thresholds. Creating if your database is being a little bit overused and we want automatically be able to hire to a larger larger instance. That is one of the ways that you can do that as well. All right, soWe also have in terms of support in terms of service.
We have, you know year year rounds experts support So if you reach out to us, we'll get you response very quickly. We're also talk to you certified so you can reallyYou can really use those cloud with confidence and you can you can you can be confident that your data is is there and that the Integrity is always preserved as well. Data recovery backup and store. We have something called the recycle bin where if you data is really if you do end up for some something happens internally or data is accidentally deleted on the UI. You can always come and recover your next recover your data in the event of accident or loss.
Fully encrypted Transit that we preserve the security of your data as well and we have role-based access control, right? And this allows you to protect a lot of your private a lot of your sensitive data data that you don't want a lot of other folks. Let's say outside of your privileged group to be able to be able to see so a lot of really key or Enterprise features inside of zillase cloud specifically that that reallyYou know, it's in pieces are there in the most Community Edition but a lot of those Enterprise features if you want them having those Cloud as the way to go. Okay. Features sort of talk a little bit about what is those Cloud can provide data quick instruction in this product. I want to talk I'll follow that up with some of the features and similar functionalities that you'll see is those cloud and this is going to be morethis will be a bit more I suppose.
A bit of a walkthrough for resilience Cloud as well. First I want to go over some Basics, right and you know, I hope I got you excited enough time in the earlier section and we'll sort of look at some screenshots here talk about what's going on and then walk through some of the different features functionalities as it was provides. But before I do that, I want to go over really some terminology as opposed first. So inside of this Cloud every single instance has a number of weekly past students or compute units associated with it. We have different types as I alluded to earlier different types of students and happy performance optimize.
You also have a compute optimize you and excuse me. You have a performance up my CU and capacity of my CU and as you can probably guess performance optimizing one is there for very very high free food large amount of queries. You want to be able to get maximum performance out of your database and the capacity optimize new is there for folks who want to be able to store more data, of course theCapabilities speed is also state-of-the-art. But it is there if you want to be able to support a you want to be able to support a larger larger collections, if you want to support more record data so good that's a concept of capacity in there and we can go all the way from one capacity in multiples of two multiples of four all the way up to 2006. Inside zillow's Cloud to keep track and to really maintain your index as well.
We also have the concept of projects. And this should be for folks who have used Cloud resources folks who have used some providers before and concept of projects should be pretty familiar each database that you have inside of those cloud is assigned to a single project and you can assign projects you can create projects as you fit to really isolate those databases. And inside each database as well. We have the concept of collections. These are essentially containers or they're very very analogous to tables and traditional relational databases.
They're very similar to collections and mongodb as well. For example DB and they're essentially the default container for vectors that are generated by the same model. So if you have a functional, let's sayI mentioned 1436 vectors on the social process all of them. Those would be those will be vectors that are generated by the same model or at least in the same domain same hyperspace, right? And as I you know, as I mentioned before we also have a variety of different support, that's the case as well. So python Java or on the way.
Sort of continuing along this line as well. We have AWS and gcp support. Azure will become soon as you can see here from from the screen here and we're also available in AWS Marketplace as well. Right? So if you have a preferred provider awesome folks refer to use Google Cloud, there's AWS. You can select that.
We have that flexibility when it comes to this cloud. We're also this four billion scale collections right for the performance optimized for compute optimized CEUs capacity units. You can go all the way up to 256 use and even with such a large amount of compute and storage you were able to get millisecond response times. And that's really the power of those Cloud right with the capacity optimized to use and those will be very very high storage capacity. You can go all the way up to 128 and that type of cluster will give you three point.
We'll support you billion vectors for you, right? So that's quite a bit when we say billion scale and we say production ready. We really do mean business scale. But also provides a very very user-friendly UI now, this is one just one of the windows or one of the different pieces of functionality is available to you inside of those cloud. The ability to take collections the ability to up the upload collections to upload your own data. Let's say in a table-based format or an S3 format and have those automatically into those platform you so if you choose to use the UI rather than using any of this in case we provide you that so that's let's move you are able to do things that that you wouldn't necessarily be able to in those communication as well.
Traveling support. So if you have your own VPC, let's say an AWS and you want to be able to connect that to you know, people connect your your instance your Zeals Cloud instance to your VPC. You have very very easy ways of doing that and we do have Guides Online if you want to if you want to if you want to create if you want to use private link, we do have Guides Online as well. And rolling upgrades and so we have you are able to schedule your own maintenance down times. I'll be able to upgrade downtown less than that one minute per month and what this really allows you to do is it allows you to focus on building your application? Yeah, you don't have to worry about any any of the infrastructure.
You don't have to worry about same upgrading to the latest version you are you have the latest and greatest Vector search experience at your fingertips with no extra. Whybackup and restore andfor a lot of organizations data loss can can often have oftentimes accidental data loss occur for variety of reasons the impropriately set permissions. It could be a variety of other factors as well. And the consequences of data loss can be pretty bad right having a reliable backup and Recovery Solution is very very important to us being able to cover data that is accidentally lost something very very easily with those types one fit solution to recover your indexes to recover your vector databases and to continue building and continue protecting World application so you don't have to worry about that. So okay, you know, those are some of the features those are some of the I guess some more important features that obseries clouding.
We do have a full list on this Cloud website as well for those are that are interested. I encourage you to go online check that out. I do want to talk very very briefly. I want to spend a couple minutes to talk about talking about migrating from illness. So I know a lot of folks here probably already have a Millis instance up and running maybe they're considering moving to zillows cloud or they want to be able to say Hey, you already have this instance.
I already managing myself. I want to go to a manager version of limits. I want to go to something a little bit a little bit. I suppose not as not as Hands-On I'll be able to focus on application developments. He provide a very very easy way of vibrating from notice and section is pretty short really just going through it's really three easy steps.
The first is you go and back up your existing build this database you can do this either directly to a local directly. You can do it to a local directory. If you have a very very large database you can actually upload it to an S3 bucket, you know of your choice backup, you know, create a backup Right Use the moves backup utility here to create a backup of your you know, this instance and you can specify either local folder or you can specify a folder on S3. You can specify location in url and it'll automatically be able to do all of that for you, right?Um this I'll get i'll get into this a little bit later. But really a lot of you know, the key idea here is that a lot of for a lot of folks that are trying to migrate from those two zeros of having there is a very very easy way to do.
So just three steps here. And in the this talk tour that I'll give a little bit later I'll sort of talking about this migration step and how you would do that as well. Okay. So now I want to spend a little bit of time to talk about some of the use cases here and I do have four prepared sort of going over some of the ways thatNeil's users have used Vector databases in production to be able to scale their applications any of these use cases areIt's not a 100% complete list by any means, but just some of the ones I figured I'd highlight so that you get a better understanding how people are using Vector databases and production. What are some of the different things that you can use this cloud with and just give you more general idea of how those talk to fit into your stack as well.
So first, you know sort of going back to the point of using sales cloud with llans. I touched upon this a little bit earlier. But this this is an example of OSS chatinism is an example application that actually we builds which allows you to chat with open source projects effectively, right? So if you have a lot of you know project information in the form of GitHub in the form of other documentation that is not readily available, you know, that's right. These Auto guess one from almost like chatbc4. There were not trained on that data actually grab that you can retrieve that project information from from zillow's cloud from OS chat on us to chat website.
We're able to really answer quite relevant questions about this. So we go through the architecture here. You'll see that on the right hand side. I have these project documents that are actually embedded and stored inside now when a person using and I was just checking voices chat application comes asks question that data gets sent to zeros first, right zillow's Cloud will retrieve the most relevant documents by doing this nearest neighbor search and it'll get injected as prompt into chat GP or whatever other llm that you decide to use and it will be able to give you a more domain specific response. Right a response to that has more domain knowledge that is more correct and less prone much much less prone to the stations.
This is a very very common pattern that we see today with those clouds using those lens and also going to sleep pretty sneak peek. Into, you know future session as well, right? We will dive a little bit deeper into this framework and we'll show you really what are some of the great ways. It's in the great libraries that you can use open source libraries, you can use to use Vector database buying Vector databases and large language models. Product recommendation this is this is a bit more of a traditional one definitely retrieval recommendation that area but it is it is a big one as well. And this is something this is an example of a new disease case that we did with the pedia where you know, you have these products and you have there's a variety of different fields, right? You have the product yourself categories labels images and be able to encode all of those into a single Vector the ability to still all that information into into a vector and store all that Mills gives you a very very semantically rich way to be able to do recommendation.
So you can imagine if I have let's say products I've users and I want to embed them that same space. I'm betting model betting strategy that same space. I can then take those store them in illness be able to get the search results you get the retriever results the recommendation results very very quickly at large scale. Have regardless of how many products I have here. Very small learning Pop Shop all the way there's somethingYou know as big as Walmart, or as big as Amazon, you have the capability using Vector search and using a vector database such as those Cloud to be able to do scalable product recommendation.
personalized search and you know, this is something that those community did with compass and again just an example of one of the many ways that you can use Vector database such as those Cloud the idea that if you have if you want to make your search a little bit moretailored to the application that you have or to your specific type of data. You can take your data and you can turn them again as you see fit into these vectors, you can vectorize them and you can be able to do much much more personalized recommendation much more personalized search through that right? So here for example instead of just searching by you know, for anybody who's who's try to search for search outside around Zillow or compost or more Redfin rather than just saying, okay. I want five bedrooms four bathrooms searching for these numbers. You can actually search by for planned by orientation my area or buy online, right? It gives you much much more powerful way of doing searchAnd last but not least reverse image search, right? So computer vision and I was very very near dear to my heart and essentially the capability to search for similar images using an image. So if I have for example, let's say very very large data set of images.
I want to be able to search for similar images in that very large data set. That's something that you can do very very easy with novice cloud andcombining that with whatever embedding model that you have there. And again, I want to emphasize that that these four are just a small sample of the wide variety of different use cases that are out there right? There are quite a few that I didn't go into. Air drug Discovery molecular surge, you know doing a lot there's a geospatial data, but these are just for that I wanted to highlight, right?Using those called the lens product admination personalized search and reversing Resurgence is in the common ones that we see and hopefully gives you a better idea of what disease Cloud can be used for what Vector databases can be used for and what some of the great ways great things great applications that can build on top of that are right. All right.
So yeah now I wanna I'm gonna stop right here and I want to give a very very brief demo or more quick tour of zillow's cloud talk about but the UI we'll go over how you can get up there create a database. What's some of the great features I'll highlight some of the some of the features. I highlighted there earlier. How those how those are implemented how you can use those into this cloud and really go over a lot of the great a lot of great functionality that we have there as well. And so stop sharing here.
andyou go to my other link down. So give me one more minutes. All right. So a quick tour a quick overview of zilla's cloud for folks who who are who are not as familiar. Those Cloud, you know, you just go to $5.
- com or if you go to zozo. com you can actually see a lot of you see a lot of those you can see, you know, exactly where to where to go from there. And again as I mentioned earlier, there areEvery database is actually insert into a project right? And if you have if you have these projects that are happening already, you can already simply go create a database us selected project there. I actually already have a database that's running in zillow's Cloud.
Once you sign on once you create an account, you already have a default project. You can just type you want to create an account right now. We will be performing some free credits to try it. It's use it as you see fit. and in the database creation screen we can see here that you know, there'sIt's pretty straightforward.
Right and I think in the slides earlier that I that I presented you actually got a sneak peek at a bit of this as well. So if I wanted to create a new database I'm not going to right now. But if I want to create a new database, I wanted to provide us a name select the project in this case. You're always looking great a new projects or to use the default project. That's there select the cloud provider select a region and then I see you type and as I mentioned earlier, we have two different types of capacity units the first days the phone is optimized unit in the second is a capacity optimized unit.
Depending on where your application needs for your application needs a higher performance higher QPS or the application needs throughput excuse me, more storage capabilities just do nearest neighbor search over a lot more vectors and then you can specify your type as well. Created you know username I create a password and then go from there. And we also provide a very very great user guide. So if you want to go if you want to get more of an introduction. It should be more in-depth introduction.
It had Visa's Cloud. I think this link here take your user guide and you receive things from there. In this test database that I've created you'll actually see you can it gives you a public endpoint that you can actually use to automatically next to your database. And in this case, what I've done here is I've actually set a whitelists to you know, actually set the wireless to be able to for anybody to access it from any particular ID, but you can change this as you see fit. So if you have if you only want let's say a very particular IP or you're feeling one, okay, so, you know a particular Network range to be able to access your record database people access your newsletter instance you can do so there.
Going back to that previous page. Going back to that previous page. There's also the ability as I mentioned as I lead to worry. It's upload collections directly through the UI and again collection is just a single units. It's a single it represents the outputs of a single embedding technology being with that way and create a collection either use your using your own data uploading your data sets or here.
We have a great feature that you can actually use an example data as well. So if you're just getting started with a vector database for getting started with those Cloud, you can import some example data and this ends up being this is actually a movie data set really really cool demo highly encourage folks who haven't used those Cloud yet to be able to try that as well. Was a you know, if you I think my monitoring is also a big part of any regular database and looking at your metrics and being being able to set up variety of alerts and being able to understand. What is the utilization. You know, how does how does that differ over a period of let's say 24 hours or an hour, right? We give you a very very easy way.
We track all those stats to be very very easy way of doing that in those cloud. You can assign more users here. All I have is the default user but I can always create more if I need to right it's like and depending on what my application requires depending on the people in my organization. That's always something that I can do there. I can create a backup, right? And again, I I lose all this earlier.
If you have data loss and data privacy is very very important to us. And you know, we've really built those Cloud around the idea of being able to serve your data and having your data benot having to worry about your vector data at all. And doing a backup here, you know being able to do backups is a part of that right to be able to create a backup here. I haven't created one. Yeah, because there's no date in this test database but it is an option for you to do so and also migration and so this is a migration that I did yesterday.
We will sort of show you a walk you through a very very quick example of a nervous migration a little bit later. In the administrative Council, I can actually see you know, this is this is there's whitelists that you can set here private link. This is actually aFeature of those Cloud online to connect directly to your VPC if you have one, right? So if you have already cluster set up, it was called instance and want to be able to connect those together. This is great way to do so you can do so for more native uses words you seeYou have resource monitors here and resource monitors alerts and we also have a recycle bin. So based on some of the tests databases that I've done in the past.
Let's say month or so or even longer than that. I can actually restore those as I see fit. These are actually stored so you can always get your data back in case of accidental loss. And then some of the more, you know, very traditional features of any Cloud Platformers. Well, you have activities alerts you have Billings.
So if you want to be able to let's say upgrade your sales Cloud instance you want to be able to use a greater greater amount of cus and settings here as well, right?Um, so he has a very very quick tour of zillow's cloud. I hope that was enjoyable. What I want to do right now is this is a database I created yesterday morning and I'm going to stop sharing here. I'm gonna switch to another window. I have a terminal that I have and I'm going to do an example migration.
So I haven't noticed instance up and running here on my Max. I'm gonna migrate that to zealous cloud and we'll sort of see what that process is. Like probably just take a minute to so won't be very long, but I just want to show you that because I know a lot of listeners here. They're actually using mobiles are already internally considering a managed service. I want that to be an option for a lot of folks here as well.
Right? So again, I'm gonna stop sharing here and let me switch over to a terminal that I have to really show you exactly how it's migration is going to work. share different screen hereAll right. So here I here, you know, I have a terminal this is this is just a you know, a quick working space that I have if I you know, I can actually show you I haven't instance of no Standalone running here. So, you know stand alone, you know, I spun that up spun that a while back and I can actually one of the interesting things that you can do. As I mentioned earlier is you can actually back up this entire, you know, this instance into a folder into a single directory and upload that into zillow's Cloud to be able to use to be able to just you know, stop your fingers do that migration there.
So what I've already done you have First Steps in migration, you know downloading them. It was back up executable is something that I've already done there. Make sure when you download those backup to download the version that is relevant to your platform. In this case. I've done the one that is for Intel CPUs and macros and it's really just a simple as creating the backup taking a look at the backup and then in here, I'll actually be using the main IO clients to copy that back up into a folder and I'll upload that folder into sales cloud.
So let me do that here. So the only thing you want to do is you know this. Back up. Use the default config, right? So I'll use back up enamel. What you want to do is you want to create that backup first.
It'sgive it a name. I'll give you all say it's my backup. And what it will do. Oh, give me one second here is I got to get the database back up and running one sec. in one and hereAll right, so maybe share that.
So what I just did there, you know, I'm gonna go was I went back into doctor desktop and I got the day base back up and running and I can actually rerun that command that I wanted to earlier. So again, it's Movis backup. And you know, I'll give it the people Emma. I'm going to ask it to create. This particular backup and again, I'll call it my backup.
And what it'll do is I'll actually go in and take all the data that you have and back it up. And in this case all the data is actually you know, it's I don't know what that was working on a little bit earlier. It's simply doing text search over some of President violence White House speeches, right?What I can do from here is I can actually take a look. I can actually use the same command and accept this time instead of create. I can actually get the backup itself.
So if I do this, you know, you see we'll give you a blur will outputs and logs at the very end. You'll see it was successfully backed up now. There's not a lot of data here. So this is actually done very very quickly. And from this point forward.
Now the back of us there. The only thing that I have to do now is use the clients copy that into a location that I like. And I've already set up and you know clients. Also I can use this name and then I'll just dump the entire bucket. right andthereit's hard buckets for local directory.
And as I mentioned, there's not too much data just a couple megabytes worth and what I can do now is I can actually take the state that I was here and I simply drag and drop it, you know, or or I specify where I want to upload it on those five, right? So I'm gonna switch back to the sales Cloud Windows it mean on it. All right. So now we are back in those Cloud. We can take a look at this application here. And I've actually already pulled up the migration window.
And all you have to do is take the back up that you just specified called my backup and upload it. Right and I can upload it here upload my backup. Give it a moment here and it'll actually should do it for you. Just like that. Then all you have to do is Click migrate.
And again, there's not a whole lot of data here, but depending on the size of your vector database might take a little bit longer. If you're very very small Vector database probably won't take won't take this long, but it's a fairly quick process very very painless. to three easy commandsto be able to grab the backup scripts dump your database and directly upload it to Zeals out here, right?And you'll see it does once that migration is finished which you know should take a shouldn't be too long here. You'll actually be able to see in the collections in this case. I actually have the data here for me.
All right, I can take a schema. In this case. I have an embedding field a longer. It's title date location as well as the original ID. I can preview the data and you can actually conduct a vector search here as you severe.
I'm not going to do one here. But it is an option for you if you if you do that necessary. All right. So again, that was a quick that was a quick intro to zillow's Cloud walking over some of the features again. If you sign up today, you will be able to get four dollars free credits to use this, but I see fit.
Migrate very very easily from the list to Zeus Cloud be able to do a lot of things be able to do exactly what you're doing with with milvis now on a managed service. Very little off so red the people who do that. Really, I think. Easily and and with very with really as you see fit, right and I'm that is that that's just the power of service cloud. And switch back to my slides here.
All right, go back in percent remote here. So yeah, so that was a quick tour because it was Cloud a very very quick demo. And again, you know, please if you are interested if this does look fun, if this doesn't look like something that you want to try if your organization is already using a vector database and you're interested in doing something to something managed something very very powerful hyperformance. He tries those Cloud you can get in touch with us. We'll help you get up and start up and running with a POC and we will you know walk through all the steps necessary with you to be able to get that done.
Right if you're interested as well. Please follow us on our social media we are at zoo is universe as I LinkedIn there as well. Approaching those Community feel free to get in touch with us through slack and there is an argument at the very bottom as well. Right, soThat is the end of this session. You know, I'm happy to look over some of the questions here and to really answer anything that folks folks might havethank you Frank.
We do have a few audience questions. So let's start out with is there an option to run dilla's Cloud on Prem?there is there is not an option to run zillow's Cloud on-prem right now on Prem is not something that we are doing just yet, butIf you would like if you don't necessarily want to run zillow's Cloud through the UI through the freezeals. com. They're always is the option of using the marketplace solution. So we are available on AWS Marketplace and it is it's one of it's one of the ways that you can use those Cloud, right? So we don't on Prem is not necessary something that we do right now, butwhat I'm trying to say that is that there are other options there that are available to you as well.
It's a super needs. Right and how similar are the Melvis and zillow's database code bases?Yeah, that is a great question. if you useif you're using the python SDK, if you're using if using the python or Java SDK with milvis, you will be able to use it with those Cloud as well with pretty much no changes required. So the code base is if you already have and it was integrated into your Tech stack, you can make a very very seamless switch into this time. And again, this cloud is based on Novus.
It's sort of understanding you can think of this in our community conditions. Those five is based off that so it is they are you you do get a lot of the power of this and the code bases are very very similar. From a user perspective as I mentioned earlier, you won't have to do anything. You won't have to make any major changes to code base. Oh, thanks so much there.
Do you have an idea when node and Ruby sdks will be available?umnot sdks, there's already a node SDK that you can potentially useless Cloud. It's not officially supported by us. But it is an option Ruby sdks. We do not have available yet and I don't know when they might be coming. But there may be you might be able to find some zealous communities supported Community Support Ruby sdks online.
I would have to have to double check but that is a great question as well. Will there be a migration path for people using milvis V1 to zilla's?So it was one dot X right. There is a migration path and I'm actually let me pull let me pull up that migration window one more time. Actually here in you know in this previous migration window if I wanted to what I could actually do is I could migrate either from movies 2 or from Nova smart. So irrespective of the version of notes that you're using to have that option, right and we do understand a lot of people are using an older version of the list for some reason and we want to be able to provide that a great path as well.
Now doing the backup and doing the data dump is a little bit different from noblest one versus MOS to and you know,just a quick caveat there, but we do provide that option and we do make it as seamless as possible. rightUm, well, let's see. What else do we have for you?What advice would you give to someone just starting out in Vector search?Yeah, also great question. I think the first thing that I would do for someone just starting out is to really understand the power of embeddings. Right? So understand what makes an embedding, you know.
What are some of the different things that you can do with embeddings embedding arithmetic Vector arithmetic to be able to do a similarity search understands in the basics first and then definitely common try out some try out a vector database such as mobiles, right? Everybody hasyou know, it might not be immediately clear but everybody has sort of different types of unstructured data that you can use everybody has data that they could Store and could query against database and I think really educating yourself on what some of those data modalities are how they fit into an application and then from there progressing throughout say download Edition or downloading the display trying or trying it out playing around with it migraine is not as you see fit. That's really where I would start right? So just send the basics first understand we're inventing is what are some different ways of doing Vector search and then start playing around with this start building Wednesday, these really cool applications that utilize this vectors that utilize these models. Awesome. So for people who maybe want a little bit more sort of guardrails to get started and what kind of projects are tutorials are available to help them get started. Yeah, great question.
And if you go to we havewe have these Guides Online one. I'm still sharing this window, which is actually perfect endless Cloud. We actually have an example. I create a you know, if ICome back to this database window. I don't have any collections here.
But for example if I well if I drop this I could but I can also import the example collection as well. Right and the example collection ends. Those Cloud will actually give you the capabilities such as little automatically import the example data for you in this case is a movie data set and you'll be able to play around with that. Without having to let's say download your own data set without having to upload your own data, right? So it's this is data that already exists in an S3 bucket, which we simply important to those cloud and you just played on play around with immediately and take a look at some ways. You can use Vector search different ways how it's Cloud be able to do that very very quickly strongly encourage.
Anybody who is interesting this kind of stuff to go. That's really cool. I think that's all we have for questions. Thank you. Again Frank for this really wonderful session.
Thank you to all of you who have joined us today. We will see you on the next zillow's webinar. Thank you.
Meet the Speaker
Join the session for live Q&A with the speaker
Frank Liu
Director of Operations & ML Architect at Zilliz
Frank Liu is the Director of Operations & ML Architect at Zilliz, where he serves as a maintainer for the Towhee open-source project. Prior to Zilliz, Frank co-founded Orion Innovations, an ML-powered indoor positioning startup based in Shanghai and worked as an ML engineer at Yahoo in San Francisco. In his free time, Frank enjoys playing chess, swimming, and powerlifting. Frank holds MS and BS degrees in Electrical Engineering from Stanford University.