- Events
Hands-On Demo: Building and Scaling Vector Search Apps with Zilliz Cloud
Training
Hands-On Demo: Building and Scaling Vector Search Apps with Zilliz Cloud
Join the Webinar
Loading...
About the session
Join our hands-on demo of Zilliz Cloud, a fully managed vector database service built on the popular open-source Milvus. In this interactive session, our AI/ML lead Frank Liu will walk you through building and scaling vector search applications with live examples.
This webinar is designed for developers keen to work with vector databases. It will teach you how to implement efficient, scalable search solutions for your GenAI apps. Frank will walk through and execute code in notebooks, allowing you to follow along and understand each step.
Topics Covered
- Live Zilliz Cloud setup and configuration
- Building a simple chatbot step-by-step
- Advanced search techniques with example code
By the end, you'll have a clear understanding of how to leverage Zilliz Cloud in your projects and hands-on experience with its capabilities. Come prepared with questions as we explore this powerful vector database through practical demos.
Today, I'm pleased to introduce to this session,HandsOn Demo Building and Scaling AI apps with SalesCloudand our friend Frank. Uh, Frank, um, if you join our eventsbefore you, definitely not, uh, strange to Frank. Um, so welcome Frank. Uh, feel free to introduce yourself and get this start. Yeah, sure.
Thank you, Stephanie,for, for that quick intro there. Um, for everybody here that is online right now, thank youfor making the time to come, uh, and,and take a look at, you know,some of the stuff that we've been building. Um, my name is Frank. I'm currently head of a N ML here at zills. I've been doing ML for probably closeto a decade at this point, startingwith the good old computer vision days.
Um, uh, and in my capacity here at Zu is a lot ofwhat I do is I get to work with, uh, users such as yourselfor potential customers such as yourself, uh,and helping you get up and running and, um, making surethat you're really, really successful. Not just on nobus, but also on zills Cloud. Uh, so today this session is gonna be more,it's gonna be more hands-on. Um, feel free to follow along. Uh, if you'd like, what I'm gonna be doing is I'm gonna berunning over some of the different featuresand some of the different things that you cando with Zillow's cloud.
Uh, I'll run through, you know, a couple of different thingsthat you might want to knowor some of the different things thatyou might need, need to know. Um, and then what we're gonna doafter that is we're going to get into one,maybe two notebooks. Uh, the first notebook is actually going to be a, a hands-onnotebook about building, building a movierecommendation chat bot. Uh, it's going to be using Zillow's cloud,and it will be leveraging retrieval augmentation. Uh, the second one is going to be about, uh,potentially the second one is gonna be about multimodal rag.
Uh, so for that one, we're going to take a videoand we're going to run, uh, we're gonna use that video. We're gonna ask and, uh,hopefully get our large language modelto answer some questions over that video. So that's gonna be a pretty interesting one as well. So, uh, I'm going to get my screen share going here,and we'll just get right into it. So, I wanna start by with giving you, giving everybody here,you know, a quick intro into Zillow's cloud.
Some of you might already be familiar with it. So if we go to cloud. zillows. com,we'll actually get a chance to, to,to really just see the interface here. I've already logged in,but if you haven't logged in yet,what you'll see is a login screen where you can, uh,where you can create a new account.
And the first thing that I didn't notice right off the batis that we have multiple organizations here. So in this case, I'm part of the Zillow's organization,but I also have my ownand my, you know, in my own organization. What I can do is I can view, click into it,view all the different projects that are in there, uh,by default, every time you create, uh, a new org, there iswhat is called a default project. All projects have project IDsand you can also see, you know, a varietyof just high level information about that project too. Before we dive deep into here, I also wanna run through someof the things that we see here on the left.
Uh, so we can look at billing, uh,we can look at the API keys that we have for our project. So for, in this case, I can look at my keyand I can also look at any members keys as well. Uh, now when, when you comeand you, you, you wanna, you know, you wanna,let's say limit access to certain databasesfor certain users, uh,or certain projects for certain users, for certain keys,this is the place to do it. So, for example, I can come in here,I can create a new API key, I can call it, uh, test two. I can, I can make them an owner, or I can make them a memberand give them access to, let's say, very specific projects.
And what that will enable me to do is,and for example here, I can also restrict accessto specific clusters as well. And what that'll enable me to do is really just make surethat the users within my organization,as I build out whatever solution that I'm tryingto build on a vector database, uh, a top vector searchthat they are capable of, um, really having access onlyto the items that they need. So I can create this test too,and you can see it's created just like that. It's very, very easy. Uh, you can look at the numberof projects that it has access to.
I'm actually, you know, for, in this case,I'm not actually going to be using this particular API key. I will be using my own for the demos later on. You can also take a look at the membersthat are a part of your organization. Uh, and it is very, very easy to invite new members as well. They get similar, you know, you can also give them, uh,access to specific projects.
And you can make, you can give, you can make them a memberor you can make 'em an owner as well. Not actually going to add anybody in this case,but that is an option, uh, for you if you want to dothat moving forward. We also have alerts. Now, these alerts are mainly what I liketo call administrative alerts. And if you, you know, uh,we can actually just take a look at some of these here.
So we can, for example, you know,if your usage amount per per dayor you know, your sales cloud usage, if that goesabove a hundred dollars, uh, what you can do is you can say,Hey, I'm going to send an alert out,and there's gonna be emails, uh, that get receivedby the organization owner. I can also come in and edit these. So for example, if I change itto a thousand right on the spot,and perhaps I want to use PagerDuty,or I want, uh, I want to use Slack as well. Those are also some of the interesting ways that we can, uh,that, that we've integrated, you know, someof these other third party platforms directlywith CI's Cloud, so that you can make it very easy to getthat data to get those alerts, uh, that you need. So I'll just update this one.
Uh, again, it's not, uh,depending on your organization's policy,what you want to do and what you need to do. Uh, these are, these are gonna be pretty interestingand some really important things for you too. You can take a, take a look at the alert history here. In this case, I don't have anybecause, um, I haven't actually set these up for, uh,my particular organization just yet. Uh, but moving forward from there, we can also look atactivity, you know, what happened here,and then we can also take a look at some settings as well.
Here I can set a system maintenance windowduring this particular window is when, you know,you'll see upgrades to your cluster or, or, or,or things that need to be changed. And these are, these are all rolling upgrades. And, and again, during this maintenance window,you will see maybe some slight jitter. Uh, you will see maybe, maybe, maybe just a little bit of a,a higher la a latency for some queries,but your vector databases willstill be up and running at all times. Um, so you don't have to worry evenfor production workloads.
Last but not least is our recycle bin. And this one you can see, you know, a lot of the previous,uh, well, I, I'd like to sayvector databases definitely spun up and then deletedor put into the recycle bin later on. So these are some of the, someof the interesting organizationlevel features in IL's cloud. Just wanna run through them very, very quickly. And what we're gonna do here is we're actually, you know,from this point forward, we're gonna take a look at someof the features for creating new clusters as well.
And when we do that, you,you see I already have a free tier cluster created here. Uh, I'm actually going to leave that there. Uh, and then what I'm gonna do instead is I'm actually goingto create a new cluster. It's going to be a dedicated one, just so that dedicated,uh, on GCP, just that we can see what someof the interesting things that we can do with a new clusterdirectly in the UI are on this page, you can seethat we actually have multiple cloud providers. Uh, it's very, very easy for us to add new regions.
So, uh, you know, we can add a new region in,let's say 2, 2, 2 weeks tops. Uh, and a lot of that is just great, is just due to ourcloud native architecture. So if any of you are familiar, if anyof you have seen my previous talks, uh,where I go high level on the MOS architecture,or if you've seen some of our documentation,that hopefully should be pretty clear to you as to why it'sso easy for us to open up these new cloud regions. Um, and, uh, uh, another, another big thing is that, uh, uh,you have two, currently two different CU typesthat you can choose from as well. First is called performance optimized.
That gives you very, very low latencyand high throughput queries. Uh, the indexing is also pretty efficient as well. And the second we like to call capacity optimized,that one really is essentially a a, a lar,it gives you more capacity at the expenseof slightly lower throughputand medium latency, depending onhow many vectors you have, you can actually sp specify a,specify the number of CU that you want. So we can actually come here to this calculator here,and you'll actually see that if we,if we have just 1 million entitiesor 1 million vectors that we want to insert, again,we're gonna be using GCP today. If you have just 1 million entities that you want to insertand you have dimensionality of sometimes 68,you can estimate the cost that that would be.
And let's just for, you know, just just to,to, to, to cover all bases. We can also look at capacity optimized pricing. So, you know, have 4 million, we can start 4 million vectorsfor less than a hundred dollars a month. Uh, and then once we go up to about 6 million,that's when we need to bump up to two cuand that's gonna cost us about 200. So, so, yeah.
So this is, uh, this hopefully gives you better,better understanding of better idea ofwhat we typically look at when it comes to, when it comesto, uh, you know, these, uh,all these different options here in the create cluster menu. Last thing that I'll note that I wanna share is the,the, the cloud backup. You can enable this directly from the create a new clusterpage, or you can enable this after the fact as well. So we're just gonna, we're gonna,we're gonna turn this off here. What I'm gonna do is I'm just gonna create this right now.
You'll see that it comes up very, very easily. Uh, it comes up very easily and very, very quickly. And once that is done, once that is done,I'm actually going to show you someof the interesting things that we can do directly on the ui. So you see the creation, uh, is actually very, very fast. Uh, and in this case, it's really, uh, it, it,it should come up to a hundred percent.
It should be done in probably about within a minute. Oh, speak of the devil. Um,and, uh, one of the interesting things that you can do,once you have that vector database upand running, you can create a collection directly in the ui. So this call collection name, for example, we can call test. Um, and you can actually just with a very, very, you,you can just, you can see theschema directly on screen here.
You can, let's say add new field. So if I have, let's say, a document ID field,and I want it to be of Char, uh,with the Max line, let's say 100. So if chars are strings for folks, uh,who are a little bit newer to zills Cloud, uh,I can just create that directly in the zills Cloud ui. So it's very, very easy for me to do, create a collection. Oops, forgot to add the dimensionality of the vectors.
I can create a collection just like that. Uh, and I can begin, I can go to the playgroundand start inserting data as well. What I'm going to do here instead is I'm actually going todrop this collection also from the ui. There's a nice confirmation notice there,and I'm actually gonna use sample data instead. So this sample collection, you'll actually see,we can add it directly into our newlycreated Vector database.
This is a collection of medium articles. So there's actually a description here. There's about 5,000 medium articles published, uh,in the year 2020. And what's going on, you know, it,it stores not just the vector that is associatedwith the title, but also the title itself. Uh, that is a string, the link, uh, the read time,the publication on Medium.
So for folks who aren't familiar with Medium, you can have,uh, your own publicationand you can, uh, you, you can really, you know, let's say,uh, be attuned to or subscribe to different publicationsand read, let's say material that is, uh, that is relevantto you and your interests. You can look at the number of claps for each article,as well as the responses as well. And what we're gonna do is we're just gonna create acollection with this sample data. And this data is beingimported directly into Zillow's Cloud. Uh, it might take, it might take a short amount of time.
Um, so what we're gonna do is we're gonna move forwardwith some of these other features, someof these other things that you can do insideof your newly Creative Vector database whilethat collection is loading. First is, we can take a look at some of the metrics. So we can take a look at, you know, CU computation,CU capacity and storage. CU capacity is at 3%because there is a certain baseline level of, of, of,of data that really needs to be stored within the cluster,uh, for, for to, to make it operational. Basically, we can expand this to,let's say the last hour or the last six hours.
Obviously, this cluster hasn't been up running forthat amount of time yet, so you won't see that much. Uh, you, we, we won't be able to see that much data, um,or we won't be able to see that much relevantdata if we do that. But if your cluster has been upand running for a while, you can view the metrics prettyeasily from this dashboard. From there, we can actually create specific usersfor this Vector database, uh,and by default there is a DB admin user,and I can create one, uh, let's call it test again. Uh, we can specify, we can specify whatever rolethat we want to give to this user.
So for example, read, write, or read only,and then we can select a password as well. Uh, I'm not going to create this particular user just here,but I just wanna show that this is another oneof the features that we have, uh, in additionto at the project level. So, if we come here and we have API Keysand members, in addition to at the project level, creating,um, uh, you know, individual users that can actually usethat vector database, we also have the capabilityof creating individual users, uh, oh, excuse me. In addition to creating project level members, peoplewho are capable of creating new vector databasesor organizations that are capableof creating new vector databasesand managing those databases. You can also go inand create individual usersfor individual vector databases as well.
So we make it very, very easy for you to really justdo anything that you need to get, uh, vector searchand to get filtered search upand running within your organization. All right, so now we have that. We can alsocreate backups as well. Um, so for example, I can add a backup, uh,cluster level backup,and it can be, you know, it, it can be a, uh, to,to this vector database that I've just created. I created one just off a spotlight like that.
So, you know, you can see it's July 24th at9:16 AM Pacific time. Um, but from there as well, you can also seethat we can create automatic backups too. So if I enable automatic backup, uh,I can set a particular frequencythat I wanna back up my vector database,uh, set a retention period. So after that retention period, uh,this particular backup will be deleted. I can set it to, you know, I can, you can set it to upto 30 days if you need more.
Uh, if you need a longer retention period,you're always welcome to come to us, uh,and ask us for that longer period,and a specific backup time as well. So, for example, we'll do seven to 9:00 PMand we'll do a 30 day retention period. Let's do, let's do custom Monday, Wednesday,and Friday with that. So we can do, we can do that just like that. Uh, it's very, very simple,and that has allowed us to just create backupsfor this particular cluster very, very easily.
Last thing here that I want, that I wanna showbefore we come back to the collect that we just createdinside the Vector database, is you can migrate betweenNovus, uh, you can migrate between Elasticsearch, uh,Anil's Cloud, and you can also,this is a newer feature of ours. You can also migrate between clusters. So for example, if I want to migrate to a new clusteror to an existing dedicated cluster, I can dothat very, very easily as well. So, comingback here, you remember from earlier, we actually tookthis sort of test collection or test data,and we've loaded this data directly into, um,our vector database here in this ui, in this screen,we connected and we actually looked at the total numberof loaded entities that are in here, uh, at, at the timethat we, that we clicked into this menu. And you can see the schema here and everything,but I think the most interesting part,and one of my favorite features of Zillow's Cloud isthat you can actually just come in here.
We have what's called a playground,and you can do insert searches, queries, uh, you can do,you know, you, you search for a particular id,you can do up search and deletes as well. So you can actually just play aroundand modify your vector database directly from the ui. It's great for testing, it's great for getting startedand great for just making sure that everything is workingas you expect, making sure your schema's rightand everything, and, and, and so on and so forth. We can also look at some of the data that is in here. So, again, because this is data from January, I believe,to July or August of 2020, uh, you know, there is some,there is some articles about Covid in here, uh,some articles about Python, um, and, and, and other,and other bits and pieces, uh, as well.
And probably my favorite feature, my favorite sub featureof this playground, uh,or of, you know, the, the, the,of this UI here is the capability justto do direct director searches from here. So what we've done here, I'm, I'm, that,that was pretty fast, so lemme do that again. What I'm doing here is I'm going to take this title vector. This is associated with, uh,the article called the Report Mortality Rateof Coronavirus is Unimportant. Uh, and what I can do is I can do a vector search directlywith this query vector.
You can also apply filter two if you'd like. I'm not gonna do that in this case. Uh, and what we can do if we search directly for this insideof Zillow's cloud is we can actually see some of the, someof the individual resultsor some of the top K results that are actually in. We can actually modify the top K two,let's modify to, let's say three. So it's a little bit easier to see.
So this is just giving us the top three results,and it's looking at, uh, it's, again,we vectorize the title, right?The title is, uh, the title vector. Here is a semantic representationof the content of the title. And if you remember, we actually took the vectorthat is directly just this, that is directly associatedwith, uh, with, uh, with this articleand we've searched for it. And you indeed, you can see the results are correct, right?The score here in this case is very, very low,and that score just means the distancebetween these two articles, uh,because we're using L two in this case, and it's very lowand, you know, it, it, it, it simply shows thatour collection is working andthat our vector search is working as well. We can also take a look at these other sort of,these other two, uh, articles in our top K as well.
And you can see that they are indeed related to Covid. Uh, the second one is called Following the Spreadof Coronavirus, and the third is called the hidden sideEffects of the Coronavirus as well. So this is just showing youhow semantic search is working in action, uh, and, and,and really just making sure that,just checking all the boxes for you in a very, very easy wayand very easy, uh, in a very easy fashion as well. Coming back here, if I wanna change the query vector,so let's say I can change this to a, you know,to a negative, I can redo that search. I think the search results will pretty much be the same.
Yeah, because I've only changed one value. Yep. So they're pretty much the same. But you cansee the score is a little bit different. You can see that the score is a little bit different.
So instead of that previous score,it was pretty much exactly zero. Now it's a little bit off zero, which meansthat the vector is a little bit more,is is a little bit further away, uh, than what it was, uh,further away from our targetvectors than what it was before. So that's an overview of really, you know,inserting you taking collections, insertingthat data directly into Zillow's Cloudand making sure that your schemaand everything, making sure that all of it, uh,is, is, is proper. Again, I can, this iswith a newly created dedicated instance. So you can just go to Zillow's cloud, you know,create a new account if you haven't alreadyspent a new instance, you'll get a hundred dollars incredits every time you sign upand just, uh, just go from there, right?So you have, it's very, very easy to do.
You, you have the capability to just, uh, to,to really just plug and play. The last thing that I wanna show on this particular screenis private link. So private link is, you know,if you ever own private VPC in Google Cloud or in AWSor in Azure, you can create a private link connection fromyour Vector database in your particular region directlyto your VPC endpoint. Uh, it is a little bit differentfor all the different cloud vendors, but we have greatinstructions here, uh, to show you how to set it up on,on A-W-S-G-C-P as well as Azure. If you do have trouble with any of that,you can always feel free to reach out to us directly.
You can file a support ticketand we'll get to that very, very quickly. Alright, um, so coming back here, oh,one last thing I forgot to mention. You can also try some of our beta features,and I'm not gonna dive too deep into this. I, this might interest some of the folksthat are here on this session today. Uh, some of the beta features include multi-vectorand sparse vector search, as well as the abilityto support binary float 16 and B float vectors.
So what, what this is doing, it, it is, uh,currently if you, if you see it when I spin up this cluster,it is compatible with Novis 2. 3. I try the beta features, you'll seethat it is actually compatible with Novis 2. 4, uh,and then later on it will be compatiblewith 2. 53 point, so forth.
So this is a great way, if you want the latestand greatest in bu, uh, if you want to use those featuresto go directly do that inside Zillow's cloud. And I'm going to, just going toupgrade to the paid version, just like that. This might take a while, while, um, it's, it's,it's really just essentially doing an in place upgrade fromM 2. 3 to 2. 4.
And, uh, once that is done upgrading, you will have access,or in this case, this particular cluster, uh, we'll be ableto use some of the new features that are INM 2. 4. So I will wait for that to, to finish. Um, but I'm, we're pretty much donewith this particular dedicated cluster here. Uh, you can, one last thing I forgot to mention.
As Zillow's cloud is so feature rich that, uh,sometimes I just can't cover, cover all of it in a,in a single session, uh, you can actually scale. So if you decide that you need more capacityfor your zills cloud instance for your dedicated server,you can scale it on demand. So I, you know, I, I've created this clusterwith one cu currently CSU capacity four. Once that is done upgrading,so once I've upgraded from Novus 2. 3 to most 2.
4,I can actually scale to two cu to four cu you know,to 16 cu to however much cu that I believe that I will need. And I can do that very, very easily as well. That scaling will also happen in place, so you will not needto change the public endpoint. Uh, it'll obviously be the same cloud regionand, uh, you know, you'll, you'll, you'll be ableto just use your, your Vector database. Alright, so now that we've gotten that out of the way, uh,I wanna show some of the different features for,at the project level.
So again, we come back, uh,we come back here at the organization level. We have these different features,these administrative tabs over here on your left. We dive into individual projects,we can actually see some moreof the different things that we can create. We've already talked a little bit about backup in therestore and some of the different, uh, uh, you know,collaborators that we can have in projects as well. We've already taken a look at this one.
As you can see, these are actually two backupsthat I just created on the spot. We can look at the creation days, uh, the creation methods. So the first one is just an auto backup,and the second one is one that I created manually. I'm going to delete both of these. Uh, or oops, I actually can't do that from here.
Gimme one sec. So I can delete,so I can delete a backup like that very, very easily, um,for manually created ones. If I decide that I no longer need itfor automatic ones, we will do that for you. So depending on the retention date that you set, uh,you'll be able to, uh, you know, we'll clear out that backupso that you don't get charged that cost beyond 30 days. You can take a look at pipelines as well.
Um, pipelines, I won't talk about this too much,but it's one of the really cool featuresthat we have here on Zillow's Cloud. One of the things that you can do with pipelines isit essentially gives you the opportunityand the ability to ingest data ratherthan ingesting vectors. So we, we can easily view the documentation here. What I'm gonna do instead, I'm just gonnacreate a pipeline very quickly. The ingestion pipeline, I'll choose, uh, again, I'm goingto ingest into, into dedicated oh one, I'm goingto call this pipeline test,and you can look at some of the different thingsand some of the different things, but someof the different ways that we can get data into our Vectordatabase using this functionality.
So again, zills Cloud pipelines, it gives us the capabilityto ingest data. We'll have an endpoint to ingest data,and that data will then be vectorized using a vectorizationmethod that we, you know, that you,that you specify on this UI here. And then those vectors will be stored directly into thecollection that we've just specified on that screen. So there's multiple different things that we can do,multiple different ways that we can ingest datainto our vector database. The first is taking documents, the second is taking text,third is taking images.
And this one right here is scalar data. So this one is, uh, is metadata, right?What we're gonna do here is we're gonna create a pipelinefor ingesting images,and we can see that there is actually two differentmodels that we can choose from. The first is VI base, uh,and then the second one is actually also a visiontransformer, but this one is trained, um, uh, using, uh,using a form of contrastive learning along with text pairs. So let's call, so we're gonna useclip, we're gonna give it a name. We're just gonna call it Tests Jes.
Just, we're gonna add that. And then once everything looks good here, you see that, uh,uh, this is the, this is the,this is the overview of the function here. We can create that ingestion, oh, forgot to, I forgotto specify a new collection name as well. So we can call this a test zero,and you can see that it has justcreated that for us directly. From there, we can create searchand deletion pipelines as well.
Um, but to do this,we can also take a look at someof the docs here via the documentation. Through this, you'll be able to see how to, how to direct,how to use these pipelinesand get data directly into your vector database,rather than having to worry about, uh, you know,rather than having to worry about, Hey,how do I do the vectorization on my side?How do I make sure that everything is okayand how do I have, um,how do I have my Vector database in the waythat I want it moving forward?We can take a look at, uh, again, these are, these are someof the different jobs that we've done. Uh, we can also take a look at collaborators for, uh,at the project level as well. So, you know, I can invite a new collaborator. Uh, this is, if you, if you remember, we can actually dothat organ at the organization level.
We can actually do that here too. Then we can take a look at our white list. Uh, this is not a white list that I recommend. Um, this is 0. 000 only use it for testing and,and debugging purposes.
Uh, and essentially that is sayingthat every Vector database or,or any IP out there has access to my Vector database. Uh, I can actually show youwhat that might look like right here. So you can see that IP address there. Um, the recommendation for production is only to use, uh,only to allow access to your vector database from very,very specific IP addresses. That'll improve security,and that'll also, uh, reduce the chancethat there is an incident with your service.
Um, we can also, I talked both about the, about the, uh,I talked already about the white list. Uh, there's also private link here as well. Uh, and we can also take a look at project level alerts. You'll see that, if you rememberthat we also have alerts at the organization level. Those were more administrative, so more related to billingand more related to, um, uh,let's say alerts that are just, uh, more around, uh,you know, the,basically managing all of your different projects.
Here we can actually seethat these particular alerts are specifically related toperformance, and so they're related to capacity, uh,to compute to QPS as well asto P 9 9, 9, P nine nine query and social agency as well. Uh, we can modify these on the spot, so we can enable some,we can, uh, delete some, uh, we can, for example,edit this one too, so we can, as longas the threshold is over 10 seconds, then we get an email. Uh, and we just make it very, very easy for you to be ableto, you know, have, have the right kindsof alerts that you need. And again, same thing aswith the organizational level alerts. You can get, uh, emails sent to you, sentto organization owners, into the project owners,or you can connect it with PagerDutyor Slack or LARC as well.
And I'm not gonna go too much over the API console here,but essentially it's a way for you to, uh, uh, it's sortof like a, a little mini postman directly insideof Zillow Cloud ui. You can think of it that way. Uh, it's meant directly to,it's meant to enable you to, to, I can select a clusterand I can, you know, I can list collections. Um, and I can, uh, you know,actually I wanna just run one right here. So here I've, I've listed the different collections insideof this dedicated,the newly created dedicated instance that I've created.
This is with, uh, this is with the restful API, uh, notwith any of the SDKs. You can see that, you can see the data that's returned here,and you can indeed verify that it's correct. So I've created a,with Zillow's Cloud pipelines in the pipeline ui. I've created a collection called Test Zero,and the Medium Articles collection isfrom a little bit earlier as well. So that is pretty much it, that is all the, all the featuresthat I wanted to go over in Zillows Cloud.
Again, uh, took me a good half an hourto go through everything. Zillow's Cloud is extremely feature rich. Uh, I did one of these,I wanna say probably about a year ago, alsoaround this time, maybe in August. And, uh, there, Zillow's cloud was great then,and it is even better now. You can, a lot of the things that you want to do, a lotof things that you want to do, you know,testing at your Vector databaseand just making sure that, administering itand making sure everything is upand running, everything is set up properly.
You can do that very, very easily, very, very quickly. You can do everything from the ui, um,and you can see here, right?And we can, you also have, if you, if you would like,you also have beta featuresthat are available to you as well. Uh, it's just so feature rich and so powerful, uh,and definitely the most performing vector databasethat's currently out there today. Uh, it's, it's, it's very exciting to see all the workthat's been put in and all the work that will continueto go into making zills, you know, making zills Cloud, uh,the greatest Vector database that's out there. So, moving forward from here, I have about 10 minutes left.
Before we go over, uh, some qand a, I want to talk, uh, I wanna actually just runthrough this, this notebook. What I'm gonna do, um, is I'm actually goingto drop this dedicated cluster first. I'm actually see that I have a freecluster up and running already. Um, there's nothing that's in here yet,but to take note of the public endpoint,that public endpoint is, uh, I've already copiedand pasted it into here. What I'm gonna do is I'm going to clear,clear the outputs of everything here.
So gimme one sec,one moment here. So this is a movie recommendation chat bot notebook. What we're going to do is show you how to perform well,to do rag over a data set,and to give that data set into,we're using open AI in this case. So here we're going to, you know, we're,we're gonna solve the dependencies that we need. In this case, we're gonna use Pi Viss to interactwith our Zills Cloud cluster.
Uh, we're using, again, the free tier here, so you can,so feel free to follow on, on directly with, uh, this,this codes snippet that I'm about to do here. Um, and then from there, we're just gonna get some importsand we're going to specify the Zillow's UIthat we're gonna use as well. So here, we've, you know, in, in, if you're unaware, I,you can actually specify keysand other private data directly in this section,this secret section here,and you can just grab it, uh, directly, uh, in, uh,you know, Google Drive, or excuse me,Google CoLab provides this user data librarythat you can use to get secrets directly inside CoLab. And from there, we're goingto use this data set from Cohere. It is a movie data set,and you can take a look at whatthat looks like, uh, as well.
So it's, it's got the title, it's got an overviewof the movie Genres, producers and Cast. And what we're gonna do here iswe're gonna extract the overview. We're gonna extract the title,and we're gonna extract the cast, the overview. We are going to store, you know, just simply as, uh,of our chart as a string, the title, the, uh, the title,the same way, uh,and then the cast we are going to store as metadata. Once we have all of that, the overview, we will vectorize,we will store the, we'll store that alongwith the metadata in Zillow's cloud.
And then we're going to retrieve the most relevantdocuments related to, uh, query that we have down here. Then we're gonna pass those on to OpenAI so that they can,so I can generate relevant information for us. So I'm gonna run this just right here. I might take a while for everything to download,but what I'm doing here is I'm reading the,I'm reading this, uh, this,this data set directly into Parquet, uh, directlyas Parquet format into Pandas. And then I'm going to usesentence transformers inparticular, I'm going to use this model.
So BG, small 1. 5, uh, it's, you know,it's a fully open source model. And we're going to, we're going to extract all the datathat we need directly fromthis data set using this functionthat we've created here, right?So it allows us, it allows to parse out all the actors from,uh, from each individual row. And we're gonna do that for the first thousand elements. You can do it for more, andit'll just take a little bit longer.
Uh, and just keep in mind, a lot of it's gonna be dependenton the model that you use as well. While that is running, we can continueand we can chat about some of the stuff down here too. Uh, there is a, here, what we're gonna do is we're goingto connect to our Zills Cloud instance. Again, that is a free tier instance I've created. Uh, you can feel free to follow along with me.
You can create a free tier instance here as well. And we're, we're going to connect directly to that. If the collection's already there, we're goingto drop it first, not just make sure we don't run into anyareas that we start with a clean slate. We're gonna create a, a schema along with that collection. We're going to add a vector.
That vector is the, uh, actually we can,I can show you what's going on here. We're vectorizing the overview. So that vector is gonna be the overview,and then we're gonna store the restof these along with it too. So the title is going to be, we're gonna store the title,we're gonna store the overview,and then we're also gonna store, uh, the cast as well. The cast is gonna be stored as an array of var charts, sothat way what,what we can do a little bit later on is we canactually do some filtering over that too.
So I'll show that, uh, once it's there. The final step, once we've gotten allof our data into zis Cloud, is to actually perform a search. Uh, here, what we are searching for is we are pretendingthat we have, uh, a user,and that user is coming to our recommendation chat botand asking, Hey, I like action fancy movies. What are some good ones to watch?We'll specify a top K here,and again, this top case, just a number of elementsthat is returned by those cloud. We're going to print all the, all,all the hits that we have as well.
Uh, before I continue, this cell just finished,so I'm actually just gonna run these very quickly. Um, so we'll see, you know,we can connect to it very easily. We'll drop the, collect the collection if it,uh, if it already exists. If it doesn't, this command won't really do anything here. We'll define a schema, create an index over it,and then what we're gonna do is insert entities intothat index or into that collection.
Again, these are things, these are onesthat I've created backup here in this cell,and you can see that that was done very, very quickly. From here, we will run this query,and then we're gonna run this search as well. We're gonna take a look at some of theresults that are, that are in there. So, the most relevant documents, the most relevant document. I'm, I'm, again, I'm searching over the overviews vectorrepresentation of, of, of the overviews.
And the most relevant documents that are relatedto I like action fancy movies,wear are some good ones to watch. They are Moral Combat Pacific Rim, and 3000 Rise in Manir. And these are the first 1000 elements in this data set. So keep in mind that, you know, as you add moreand more movies in there, you'll see moreand more relevant results too,just based on the overview, based on the model. And using Zillow's cloud, I'm able to understand what,or I'm able to get my, my c my rag system to understandwhat is, what the content of the movie is itself,just based on the overview.
And I can recommend some good ones based on aquery like that as well. Another interesting thingthat I wanna demonstrate here is thecapability to do filtered searches. So let's, I'm a fan of Ian McKellanand I'm a fan of, you know, action movies from, uh,from him in particular. I can do a filter here. So, um, this filter is, I'm just using the array contains,uh, metadata filter that is a part of Zillow's cloud.
And if I run that, you can see that it actually gives me,Lord, the Rings return of the King, uh, whichthat is a movie that does indeed have a inMcKellan as part of the cast. And we're gonna see the overview here as well. Now, from there, we can just give the queryand we can give the context directly to OpenAI. Uh, in this case, we are using GPT-3 0. 5 turbo, uh,and we can just look at, uh, some of the interesting one,or we, we can just look at what the response was, right?So based on this right here.
So based on our latest search, based on the prompt,based on the results of that search, uh,this is the response from our movie chat bot. So you can see it's very, very easy. We've done, you know, just in really a couple of cellsand not that many linesof code within a hundred lines of code. We've created at least a prototype of this chat bot. Uh, and we're show we've, we've showed how it's capableof getting really, really relevant information to our users.
I only have about two or three minutes left here. So the last thing that I wanna show, I'm not gonna runthrough it fully, is a multi modal rag demo. This one, currently, this one is currently set upto use viss, but it can easily use Zillow cloud as well. I'm not gonna talk too much about this,I'll just run through it very, very quickly. I think everything is actually,has actually been run here already.
And what we're doing here is we're taking a video,splitting that into framesand audio, embedding all that together in the same space,and we're retrieving the relevant results. So one of the other things that we can dowith Zillow's cloud, rather than working with just text, isto work with images and to work with audio as well. So I'll just show you the results. I won't show you, uh, how the soft stitchis made here, so to speak. But we have a prompt what was weird about the coffee mug,and this prompt is related specifically to the videothat we have inserted into, uh,into our Zills Cloud instance.
The video I'll actually show you right here is the Google iopre-show in it at the very beginning, the musicianwho crawls out of this big coffee mug. So we can actually see that here. What I've done is I've taken this whole videoand I've taken, I've split up the frames and audio,and I've stored it into, I've stored itinside Zillow's cloud. But coming back to this demo here, we can actually seethat the most relevant content that was retrievedby our vector database, it's related to this prompt,is this image right here. So what was weird about the coffee mug is this image righthere, and you can actually see that indeed, that there is,you know, there's something going on here, right?So that the, that this is indeed very,very related to our prompts.
You can see actually the musician is crawling outof this coffee mug. And based on that, we can eventually,I'm not gonna show it here, but we can give it, uh,we can send it into, uh, a large language modelof our choice for this particular notebook, I choseto use Meta's open source model called Chameleon seven bLama 3. 1 or 3. 2 eventually will havemultimodality built into it as well. So that's something to look forward to too.
So yes, that's it. Um, I want, you know, again,just by way of a quick summary. We showed today how to take a Zills Cloud instance. We showed how to, uh, you know, how to, howto really modify our, how to create a new vector database,add a collection into it, modify it, you know, have allof these, uh, these, these,these really enterprise features,these quality of life features around it. We showed how we can easily build a rag chat botusing Zillow's Cloud.
And also we've seen the potential for vector databases to gobeyond text and to go multi-model as well. So I hope this session was helpful for you. Uh, I will take a look at someof the questions from the audience. Nowwe've got some AI assistance here that's very cool. And from Bill Reynolds has PSA Carbine is gonna release aZillow's native analytics suite that is smart about Gen Aquerying and JSON objects.
Uh, all the topics that Frank covered today, super easyto apply Zillow's features, um,and reach out to support@carine. comto request an early invite. So if those, that's something that interest folks, folks,definitely be sure to reach out to Bill as well. Uh, so if you have any questions related to someof the topics that I've, that I've spoken about todayor some of the things that, um, some of the thingsthat are related or adjacent to this field as well,feel free to reach out to, to me or reach out to us. Um, we will make these notebooks availableto all the attendees, uh, after,after this session is done here.
And until then, um, Steffi, I'll hand it back to you. Um, thanks Frank. If we don't get any further questions, um,we just wanna thank Frank, um, for this sessionand I'll see you guys sometime soon in the future.
Meet the Speaker
Join the session for live Q&A with the speaker
Frank Liu
Director of Operations & ML Architect at Zilliz
Frank Liu is the Director of Operations & ML Architect at Zilliz, where he serves as a maintainer for the Towhee open-source project. Prior to Zilliz, Frank co-founded Orion Innovations, an ML-powered indoor positioning startup based in Shanghai and worked as an ML engineer at Yahoo in San Francisco. In his free time, Frank enjoys playing chess, swimming, and powerlifting. Frank holds MS and BS degrees in Electrical Engineering from Stanford University.