- Events
Unstructured Data Processing from Cloud to Edge
Webinar
Unstructured Data Processing from Cloud to Edge
Join the Webinar
Loading...
About the session
In this talk, Tim will do a presentation on why you should add a Cloud Native Vector Database to your Data and AI platform. He will also cover a quick introduction to Milvus, Vector Database, and unstructured data processing. By adding Milvus to your architecture, you can scale out and improve your AI use cases through RAG, real-time Search, multimodal search, recommendation engines, fraud detection, and many more emerging use cases.
Edge devices even as small and inexpensive as a Raspberry Pi 5, can work in machine learning, deep learning, and AI use cases and be enhanced with a vector database.
Topics Covered
- Introduction to Unstructured Data Processing
- Introduction to Milvus
- Adding Milvus to Your Infrastructure
- AI Use Case Improvements
- Edge devices working with AI and vector databases
Today I'm pleased to introduce today's session,unstructured Data Processing from Cloud to Edge. And our guest speaker here is Tim Span. Tim Span is the principal developer advocate at Zillowwith over 12 years of programmingand data engineering experience. And before joining Zillow,Tim was a principal developer advocate at Clouderapromoting open source data engineeringand streaming technologies. And then previously he was also a developer advocate atStream Native, a principal field engineer at Cloudera,a senior solution architect at Aris Data,and a senior field engineer at Pivotal.
He blogs for DZ Zone, where he's a big data,a big data zone leader,and he runs popular meetups in Princetonand New York City, um, on topics like machine learning,big data, deep learning, et cetera. He's also a frequent speaker at conferences likePulsar Summit, data Summit, summit Con 42, et cetera. Um, and Tim holds a Bs, ms,and BS degrees in computer science fromMontclair State University. Welcome Tim and get started on your topic today. Very awesome.
Thanks for the, uh, introduction. Let me, uh, share my screen here. Hopefully all that is working fine. It's always, uh, always a mystery,especially when you've got a coupleof different screens here. And I've got, um, broadcasting from my arcade center here,so it's a little, uh,different from my normal broadcasting booth,but, uh, this one has more arcade games in it,which is kind of cool.
Hopefully they don't automatically start up. They, I have had Pacman start up in the backgroundbehind me, so we'll find out. Hopefully not, uh,I don't think I have to do the intro there. Uh, so this one is pretty straightforward. This one has been rapidly evolving aswhat you can do at the edge has been rapidly evolving.
If you've, uh, watched the, uh, AI news,which changes every three to four seconds,maybe quicker than that, uh, there's been some, uh,new models coming outand I've downloaded one with Alama yesterday on a new devicethat I got two days ago. Uh, my friends at Nvidia have, uh, uh,got me a device hereand, uh, we'll show you some demos on it'cause stuff just works and it works really fast. I was pleasantly surprised how quickly you can,uh, just get these things done. But, uh, yeah, we'll do introduction. We'll show some edge cases,we'll show some demos, do a bunch of stuff.
Now, previously in former lives, trying to, uh,save things, especially things that were unusual data. I mean, they're not unusual now. I mean, think of a use case for Gen AIand the new AI use cases that's not using big text,interactive text, images, videos, sound, all kinds ofslack channel, all kinds of weird stuff from everywhere. It was crazy. Where do you store it? How do you retrieve it?What do you do with it? Well, you can do a lot with it.
Uh, while traveling in the field with my vis bird,we found you can run, you know,these vector databases anywhere,and there's data everywhere, even unusual places,sometimes not connected to even the internet,and you gotta do what you gotta do there. So we get all this crazy different types of data sometimes,uh, interconnected. What I've seen coming from traditional data engineering,more structured data stores, even in Hadoopand, uh, S3 bucketsand object storage, being able to connect this, uh,richer, unstructured data with structured data isn't hardwith something like vis'cause you have the ability to add metadata, J fields,different attributes. So I could put the, you know,I could link whatever data I have, say it's a picture,put all the metadata relevant to it. Also put hey, if, if it's connected to an Oracle recordor if it's connected to a Mongo DB recordor whate any other database, which you could have, you know,a bunch of databases for, you know, relational dataand more, uh, structured data.
But with vus being able to store these quicklyand being able to get them back justas fast as pretty amazing. I'll show you on the edge, it's a little different. And the flexibilityand the different deployment models make for a lotof interesting use cases, especially different size boxes. You know, you could get a $10 device,or you could have a several thousand dollars devicethat's got 64 giga ram and terabytes of storage. The edge isn't just a, a tiny little button, uh, sensor.
It could be an edge server, it could be, you know, oneof these powerful Nvidia boxes. There's a lot there. So we get all these things. We use a deep learning model to get these encoded rightwith the proper embeddings,get them into our vector database, get 'em indexed,and then, you know, have them stored where they needto be stored and ready for retrieval instantly, even ifthat is running right on the edge here,right on your, uh, device. Again, there's a lot of data out there.
It's just gonna get bigger. And there'll be more sources of this. I mean, real time video feeds are coming from eyeglasses,you know, smart, uh, cars, smart devices, smart shirts. I had a use case I was working with, uh,someone within a dangerous space. They had a smart shirt that had a live camera.
It was checking for thermal, it had sensorsfor oxygen levels. So in case, uh, the person was in a dangerous location,you'd get that instant feedback to know,Hey, dangerous stuff's going on. Uh, so there's a lot of different ways to do things,but why use a vector database, especially in the use caseswhere a lot of the stuff is on the edge,or it's communicating from the edge to maybe a, a cloudor a central, uh, cluster. Obviously being ableto do high performance search is reallyimportant for a lot of use cases. But when you have a vector database, you can also do thingsthat you expect to be able to do in a database.
You know, create tables, collections, uh,insert records, read them back, do updates, delete data,make sure everything's fresh. Uh, make sure you get your, uh, results, you know,immediately make sure things are stored even if youhave to shut down, if things restart, if you lose a node,things that happen in the real world,you wanna make sure this data's always available,you know, regardless of what happens. You know, even if, uh, a node goes down, you know,being able to handle those things, being able to scale upand add additional, uh, storage, add additional query,being able to support more users, more data,more collections, more applications without having to worry,you know, am I gonna hit a wall sometimeor I'm gonna hit a a cost wall?The great thing with VUS is open sourcescales out as big as you need. It's not, you know, a limited featured version. There are people using open source visaround the world in very large form.
And obviously being able to manage your data,which means you gotta be able to ingest it,index it, query it. And if you're not doing backups, I mean, yeah,this is unstructured data. Maybe you don't think of it as the same way you didwith your, uh, relational data,but backing up, migrating between versions, migratingbetween, uh, different environments. Like, uh, I'll show you, uh, backing up from oneof my devices, and then I can load that into the cloud,you know, and that can be automatedor built into an application. But those things are important,and that's why you have a databaseand not just, you know, some library somewherethat maybe stores something, maybe saves it.
And you need to be able to operate it. You know, if you're doing a toy use case,I've got five documents, I wanna do a chat on 'em,and maybe it's just, uh, something you're learning,maybe it's not that important. But once you get to the pointwhere this is something people rely on,and believe me, when people see the power of being ableto do searches, take an imageand use that to search with being ableto do lookups against text, real fast search things. People will love these apps and they quickly expand. They quickly need operational, uh, concerns.
People will aware where are we gonna deploy this?And being able to having those options,whether it's in different, uh, public or private cloudsor in your on-premise, uh, infrastructure or docker,or, you know, use Zillow cloud. If you want a fully managed environment, obviouslywith any database, anywhere you have data,you better be able to monitor it for, uh, healthand performance concerns. So, you know, things are running well,especially you get enough users, you know,in the beginning everything is subsecond, then allof a sudden you add, you know, 10,000 more documentsand a hundred more users and things slow down. You need to be able to, uh, monitor what's going onand know when you need to expandor maybe, uh, add indexes, change indexes,and the community is always out there to help you. If you're, you know, started somewhereand you're a little stuck,or you're having problems with, uh, performance,definitely reach out.
We'll give you all those different channels to reach us. Obviously, when you have devices deployed, I mean,you're not gonna just have one device sitting in onelocation, all of a sudden it's in all of your trucks,it's in all of your offices, it's in all of your, uh, ATMs. These things quickly expand,as I've seen in iot use cases from manufacturingto oil drilling. Uh, you could be in thousands of locations,maybe running on different desktops for different purposes,like, uh, securityor monitoring, uh, being able to support a lotof different users and apps concurrently with security. If that hasn't come up yet, uh,I don't know you have a production app yet,but that's, that's important.
Uh, and the features that VIS makes that, uh,really possible is having a very elastic architecture. We won't go fully into that. There's a lot of articles on that. When I saw that, I was impressed. It reminded me of the optimal, uh, Kafkaand Spark, uh, architectures where things are distributedand things can scale up independently.
You have a separation of compute and storage. It's the, you know, design principles you expect in anenterprise level system, being able to support a lotof different indexes and more always on the way,because data changes. And as you see these models improve, new ideas come out,there's always new indexes,and VUS just finds a way to quickly incorporate themand make, uh, use for them and make them available. So you could try different things. This can be important too on, uh, edge deviceswhere you might want one that's maybe a little faster.
You know, you got, uh, that is an option there. Lots of different ways to do a search,and one that's coming from a traditional data stores,the ability to tune how you do consistency,uh, that's pretty cool. I mean, it's usually you're, you're stuck in capand whatever's decisions were made a long time ago,those are the ones you're dealing with. You've got options here. Uh, you may not, you need that for most use cases,but it's, uh, nice to know you have it,especially if you're doing some, uh,weird edge cases and such.
Again, support for a lot of different hardware,accelerated compute, and there's a bunch of them out there,and they're all in the database, which is cool. You mentioned lots of different deployments. Uh, all, all the indexes you expect to have binary,sparse one's for GPUs, specifically for Nvidia,and I'm using some of those on one of the Edge devices, sothat seems pretty good. Um, lots of different ways to search. I mean, it'd be nice if there was only one way to search,but there is a bunch of them,depending on if you have sparse, you have dense ones,different range, uh, searching multiple vectors,grouping things together, filtering your datawith all those metadata fields we mentioned,so you can really tune what results you get back.
Lots of options here. Uh, this makes for a really nice solution for the edge,but also for wherever you're running. Amil, thiss is really a general use vector database. Even if you don't have gen AIor AI use cases, it, it, it solves a lot of problems. I've been, uh, really happy with it.
This is some of the devices on my desk there. There's a lot of them, but, uh,edge AI is just gonna keep expandingbecause as we push automated systems, robots,people enable smarter cities. So things will just work. Uh, you need sensors and cameras and audio,and to be able to have localand remote in, uh, you know, some kind of federationwhere you could find results you need right away. See if something's wrong, like, hey,there's a leak in a pipe somewhere, you know, be able to dothat and then detect those things quickly.
You know, there's a gas leak, there's a fire somewhere. Things you wanna know right away. Being able to have, uh, AIand, uh, vector databaseand the ability to process, you know, datathat you need for the real world. I mean, unstructured data is real world data. I have a picture of what's in front of me.
I've got the audio of a fire,I've got a video showing something going on. Operational things you gotta worry about. Safety, those things are very important for, you know,as we improve cities. The same thing for manufacturing. You know, has there been a, a defect?Being able to figure that out right awayand being able to store locally, uh, all the, sayfor one particular machine, what are the defectsthat it's had in the last thousand hours, thousand years?Whatever it is, as much data as you wanna store.
You could store a lot of it in these new, uh,edge devices now, like that, terabytes. But even if you just store the last thousand errorsof good data, so when an image comes through,I could check it and yep, that's still good. Uh, a defect, looking against known defects, Hey, this is,uh, related to an error we had two weeks ago,and we can connect, uh, look up the connected data,and I could push it to, uh, a local operator, push itto a Slack channel messages,and then I can find outwhat exactly is going wrong immediately. Maybe even solve it automatically with ai, uh,if you've got the right, uh, model and algorithms. But the, the first step is detection, being ableto search quickly and do that locally.
So maybe I'm in a secure facilitythat doesn't allow me to leave it. So I can have a small edge server,or I can have a powerful edge device like, uh, Nvidia Orinor something sitting right there that can find it,detect it, check it against models for errors,and also check the, the vector data store locallyto see is this related to something that's a problem. Maybe it's not. Maybe it's something we've seen beforeand people have got involvedand say, no, don't worry about that. When you have autonomous cars, which, you know,we're getting close, you know, there's, there's a lotto go on there, but having a vector database in therethat knows what people look like, you know, there's a lotof great sensors and lidar and camerasand detection algorithms,but, you know, having a, a fast data store that I could getto and not worry about,even if there's no network connectivity to know,Hey, this came up.
Do something. And being ableto get those results right away. Retail, obviously, having, uh,to check out your own stuff isn't everyone's favorite,but, uh, having it smarter and be able to search things. Pretty cool. And again, doing the same thing in your house.
Being able to know everything in your house is, is that, oh,that's Tim's cat again. Okay, that's Tim's cat again. Just take a picture. You don't have to, uh, change things. It's the cat.
Don't worry about it. Oh, I saw a quick question. Maybe I take a quick look. There are search types directly linkedto the corresponding indexes. Um, yeah, we'll get into that,but, uh, yes and no.
Most of the searches support most of them,but obviously performance,having the right index makes everything better. Uh, it really depends on your data,but, uh, most of the search types are supported by, uh,most of the indexes. I I'm trying to think if there's any that aren't. I don't think so, but we'll, we could take a look at that. We'll get you a deeper, uh, answer on that later.
Uh, so cool thing. Again, I've got a couple devices running right next to me. I've got a Raspberry PI fivewith the new AI kit added onto it. So it does a faster AI inference,and I've got a Nvidia Orin running right atthe edge of my hand here. Uh, they let you do a lot locally.
Again, being able to have the local search right there. Uh, if you've got things that are proprietary documents,things that are secure, thingsthat you not risk having on an open network,maybe it's privacy issues, you know, you wantto keep things private, uh,but you still want to be able to run applications,run ai, get things done. Well, you could run that right on that device. No one else has to see it. Everything can be encrypted.
SSL all around it, maybe even no network on that. It might not need it. It might just communicateto a local screen or to a local device,or maybe just pushes out, you know, I send my datato a secure Kafka stream,or in my example, as you see, I'm pushing thingsto my own Slack group, uh, being ableto do on device object detection, lotsof different AI things, especially if you've got, uh, oneof the NVIDIA ones, which supports a lot of small, mediumand large, uh, language modelsand all kinds of models on there for pretty much everything. You go through that listwith the Nvidia nims, and it's amazing. Now, being able to run vis light on most of these devicesin the latest version, it's just a Pippin install.
I can run the full local database there. I don't need to run Docker. I don't need to run something a little bigger,which may take some of your, you know,really important resources. Now, vis light is light. Uh, I don't want to put, as you could see, you know,keep the number of vectors in there.
Not super huge, but, uh, we can back them upand send them somewhere else as needed. And you could also have those refreshed, keep them, uh,just the most recent ones,and then send the older stuff up to, uh, the cloudor distributed another, uh, avenues pretty easy. Now, what's, what's making Edge better?I mean, I call these the hyper enablers,and it's just things that are making this a no brainerthat you're gonna have them 5Gand there's networks everywhere, and they're getting faster. I could tell you driving across the country, maybe notas much in Wyoming and South Dakota,but it's more than it was two yearsago, I could tell you that. And those are just gonna be around every city.
You are gonna have a lot of bandwidthand it makes sense to, you have, uh, devicesthat can take advantage of that and produce dataand capture data right there. iot and the number of sensors have gone down in price. The number of types of sensor. lidar used to be something for, uh, the richand famous Lidar is now a hundred bucks, and there's moreand more sensors. I mean, I've got a hundred sensors on my desk.
There's so many cool things you can capture them. You see when, uh, things are going on,like when there was air pollution, to be ableto have local pollutions that, uh, sensors that could checkfor different pollution levels,different ozone levels, important things. Being able to tell when, uh, things are going on,detect fire, smoke, all those things are important forso many use cases, and those arejust getting cheaper and easier. Computing power just keeps going up. Uh, raspberry pie used to be pretty cute,and you had half a half a giga ramand you had A-O-K-C-P-U.
Uh, now you can get eight giga Ram in a raspberry pie. There's other, uh, raspberry pie equivalents with 16 and 32. I've got 64 on an Nvidia, I've heard of 1 28 even more. So you're getting more ram, more CPU power, full GPUs,TPUs custom, uh, asics in there to give you a lotof compute power and more storage getting there. I mean, it used to be,maybe I had a couple of gigs on there.
I can have terabyte drives. The new Raspberry pie supports real SSDs. I mean, you could put serious amount of storage affordablywith low power usage almost anywhere,and that's just gonna get moreand more faster, better every year. Being able to have really powerful neural networksand generative ai, large language models right there,running and running fast. A decent number of, uh, framesright there is opening a lot of interesting use cases.
Some we haven't even thought of. Uh, being able to do unstructured data processing,capturing all of that,and having a database that enables you to index thatand search that at scale very quickly, even,even if it's on a phone. I mean, it, it's, it's incredible what's going onand it's just gonna get more, obviously really important. Unstructured data is in the real worldand where, what's in the real world, these devices. So vision, getting imagesand videos, getting audio from all the camerasand microphones, getting raw text.
It could be spoken, it could be captured from, uh,different input devices, uh, local things going on,or even produced by the local model. Again, having those neural networks, all ofthat right there at the edge, prettypowerful, pretty important. Uh, I, I'll show you the demosafter I finish the slide. So in case we, uh, in case something uh, goes wrong,uh, I'll, I'll have a little time to restart stuff of it. Working on an Edge device.
Yes, I am. Okay. Uh, maybe I should show it now. Okay, let's, let's show your real worldand hopefully people don't mind. Command line.
I'm gonna show you a little command line, justso you know, it's a real edge device,otherwise they're like, is that a real edge device?Uh, maybe it is, you know, this is,yeah, this is a,my local device here. Just show you the, uh, horsepower here. 64 giga ram, A lot of CPUs. This is the Nvidia. And on here I am running, uh, a vis,uh, server in Docker Compose, as wellas if you see this little do tvb,this means I'm running Vis light.
Uh, so I've got three waysof running on this particular devicethat I got in the mail yesterday, which thank you to Nvidiafor having this supercharged and ready to go,because it didn't take, it took 10 minutes to get upand running, and that was just installing a couplelibraries, and I put some extra ones on there. Uh, so I can run, uh, a local,I have a camera pointing to some stuff on my messy desk,and I can run thatand run it to, uh, the cloud, which is,uh, Zillow's cloud. And while that's running, we'll go up to Zillow's cloud. And you can see I've got a couple different collections herein my, uh, serverless environment. Uh, one of them is Orin Edge.
This is the one that is getting called from here. This is an Orin. So what this app is doing,and I've got the full source code out there. I've got a camera right here,is looking at the junk on my desk,and then it goes into an algorithm to, uh,give you a description and a nice caption there. And then, as a way to segment it, obviously I'm sending thisto my, uh, vector database,and you can see I've got a couple of them here, uh,that we put in recently.
I've got a couple little metadata. I put that caption in there. I put the file name of where that image is. Uh, I could put that in S3. I could put that in Minio right now.
I have that locally trying to figure outwhere I wanna put these images. That is always a thing. Or do I keep 'em for long?You know, that is a decision based on your storageand your apps and what you wanna do with it. Obviously I have the vectorization of that image,but yeah, you know, with the metadata. But it's good to have that.
So that version got sent to the cloud,so it's running on the edge here. We had a network, I sent it to the cloud. Now, I also have on this, uh, machinerunning is Vis and Dr. Compose, you could see it's the GPU versionbecause I have a pretty good GPU here. Like it's, uh, it's better than my laptop.
I think I'd rather use an Orin than a,than a Power book, though. I don't know if I wanna run, uh, my slides off, uh, there,but I could, okay, so we can runand send it to that, uh, local Docker version. Now that Docker compose version of Melvathat's running on this Nvidia hardware here is,uh, connected to my, uh,I put connected an at two instance. If you haven't used at two, it is a really awesome ui. It's probably the, the bestfree UI I've seen in a, uh, in,uh, in the open source.
I mean, it is pretty nice. Uh, I like this, uh, ui. So, um, we'll show you how this works. So we've got, uh, all the collections on here. I'm logged in.
I'm using 2 4 6. I've got a GPU edition here. I only started this today. I, I thought I should probably have this on there. So I added this.
So if we look, I've got a collection herethat's loaded only five rows in here. You can see I've got, uh, uh, indexed, uh, vector,got time caption, couple fields. Now I could do a search here. I could also filter on the different fields we have. So say you wanted to filter on, if we had a lotof data on the caption, maybe one that just says laptop,I don't know, we could do that.
Uh, caption. Is itlowercase laptop?Yeah. Uh, we could search that. Only two of them had that. Again, that's running on the device along with this ui.
So you could see the data there. Um, every time I run one of these, as you see here,we get the result in the command line, which if,if you're doing this headless, you're not even seeing that. So I make sure I send those results to my Slack channel. You could send it anywhere. You couldsend it to a data store.
As you can see, I've got a lot of weird junk on my disc. I've got a two of the mil, this birds, I got a cat,got a button, got a little, uh, Pacman game. I got the big one behind me,but it's too messy to show you all that stuff. It'll break the internet. We don't wanna do that.
So as we run it, we got it uploaded thereand we've got it available. So you could see it in, uh, in the display for my, uh,for my local, uh, Viss running in there. So we've got, uh, the third way,if you look at my different runs is just a regular run. This is running to that local Orange Edge ai,that's my local, uh, Viss light databasethat just needs Python, which is great, you know,so you could just run it, you don't need anything special,but that's only stored in that local little database. And there's some things we could do withthat local little database, like you can, uh, search it.
So I can run my, uh, search against that. Oh, is that gonna pop up a screen? Yeah. Uh, I don't have, I don't think I wanna pop that up. That pops up on another screen, which you won't see. So we won't do that one, but we could do a searchand it gives you the resultsand then it posts them to, uh, to,uh, slack channel.
Good way to do things remotely. So that is a couple of devices running. Uh, I'll show you the Raspberry Pine one as well. And they give you links to the source codes if you wannaplay around with it on your own. Now, uh, on, uh, my device,I can, uh, connect a notebook up to Zillow's cloud sothat I can, uh, search those ones.
So that's a great way, if you have data that's remoteand you, uh, want to be able to access it, pushing itto the cloud makes a lot of sense. Very easy to access through either the UIor through a notebook or through anything else. Uh, I'll show you in the slides how easy it is to pickbetween, oh, I'm gonna do mil this light,I'm gonna do Docker. Gonna do a regular little standalone, uh, instance,a little cluster cloud. You're just changing your, uh, login.
It doesn't, nothing has to change much. It's pretty, pretty cool way to do that. But let's show you the raspberry pie. Uh, the raspberry pie I have running over here,and this one is also have a llama running on there,but, uh, I don't have the screen hookedup, so I don't wanna show you that. So what we're doing over here is we're doing, uh, pose, uh,estimation using the halo, uh,raspberry Pi AI kit, which enhances the, uh,abilities to run, uh, some different, uh,ML deep learning and AI use caseson these affordable raspberry pies.
It is a nice enhancement that is hooked up to a camera nextto me, and that will run, uh, uh,posed in quiet mode. It's still a little noisy 'cause I got some errors in there'cause it's, I turned off the screenand it doesn't really want you to turn off the screen. Uh, it found me, found my eyes. Dunno if it found the bird's eyes to find the cat's eyes. Yeah, so this is running against, uh, what's over there.
Yeah, this is also doing a similar thing,but this one is just connect connecting to, uh, the cloudbecause they really don't have thestorage on there to do much. I can run it in vis, uh, light as well,but to just to show you some options. So that is the, uh, raspberry pi, uh, pose. And it's just, I'm just grabbing, uh, leftand right eye coordinatesand the confidence that it found eyesand just storing it there along with the vector. And again, ooh, that's creepy.
That is a blurry picture of the vis,but it found its eyes, uh, pretty special. But, uh, yeah, if, if I hold it, uh, close to my face,you get the, you get the right results. But just to give you an idea there, yeah, so we can run ona raspberry pie and you don't need that extra AI kit. It makes it nice so that you can, uh,you know, do more edge stuff there with a little power. Uh, they're still working on some of the libraries.
It does the basic stuff like pose, estimations,segmentation, object detection, uh,for an inexpensive device. It's not bad. Uh, I'm, I've been pretty, uh, happy with itand all in Python, so it makes it easy to write applicationswith, uh, with VISand whatever other things you want to do there. Again, I also have an older Jetson Xavier NXthat also runs Vis,and I've got, uh, some articles on that one. The Orrin article hasn't come out yet 'cause I got that.
Or yesterday. Uh,and we have this webinar,so there's some other stuff going on. I also have, uh, some working with some smart cameras, um,working on, uh,that if people are interested in those use cases. If not, let me know. Oh, uh, see, uh, is that the Vis ui?Uh, those are all, lemme go back in there.
This right here is the, is an open source VUS ui. This is called A-T-T-U-U attu, I call it. I don't know if that's what it's really called,but that's what I thought it's called. Um, this is the local, uh,local one running on, uh, on the, uh,Nvidia, or that is not the ORN device. Uh, they have a, a lot of UIsand some pretty impressive stuff.
I'm not running those yet. That'll be coming probably next week. I didn't have time to put that in here. I'm still trying to map out a nice application there. Uh, yeah, so this is an open source onethat works with vus.
Uh, you don't have to use this,but it's nice if you are using zills in the cloud,which there's a lot of free spaceand free availability there. This is the UI for that one. That one's pretty nice. Uh, this is Jupyter Notebook, the same Jupyter Notebook. This one's running on my Mac.
It also can run it off of the Raspberry Pieor off of, uh, the, uh, Nvidia Box. Uh, run, run server. Run local. Yes, that is, that is my really sad, uh,way of running, uh, different applications. 'cause there's three different apps here.
See, there's Orin Cloud. If I look at Orin Cloud,and I'll post this source code, you could see here,this is connecting to, uh, my specific, uh,serverless URL at Zillows plus taking in a token. So Run Cloud just runs thatand passes into token, uh, Orin by itself. This one is, uh, this one is connectedto the database. And if we look at the database, this is our,our local vis light.
So Run runs that oneand then run server runs the orange server,which is also running. That is, that's my local ip. I could go by name that go by something else,but this is the visthat's running in the Docker composed here. And as you can see, uh, it's got et CD, it's got minia. I guess I could store the images there.
Um, and there's at two, which is the, uh, ui. Again, you don't need to use the ui,but I think it's pretty awesome looking. And it lets you, uh, look at all the, uh, schemas, uh,do a vector search, browse your data, also do stuff like,uh, backup, export your data, uh, give you example,source code for doing a search. I mean, uh, it's pretty cool tool. Uh, so those are what the tools are.
Uh, let me get back in here. Hopefully that answered your question. Uh, so we're running with a lotof different hardware examplesand there's more people are coming outwith different hardware, lots of stuff there. Again, we mentioned, uh, some of the reasons why you wantto use a Vector database on the edge. As you could see, flexibility of where you send this data.
I mean, running it, uh, mils light is the easiest. Takes up to less resources, but the data's just there. So if you want it somewhere else,obviously you could send it somewhere else. You could do, uh, send it to multiple locations. Uh, what we used to do when, uh, Hadoopwith dual ingest, you could do that.
Uh, you could also do backup. I'll show you how to do that. Having instant local search for whatever's on that box. I mean, if I'm running a, uh, Nvidia box,I'd be capturing a lot of images, texts, uh, logs,maybe sensor results. I've got some sensors for the, uh,Nvidia, whatever's on there.
Being able to instantly get that data backand not have to go over a network again,not needing a network, being able to do your own local, uh,rag, uh, solutionsand being, making your Edge AI even, uh, more powerful. Uh, and we can run a lama there. Ton of different things that could run on the, uh, Nvidiabeing able to do local live video processingand not have that, have to go to the cloud. Oh, RPI, yeah, that's, sorry, from my stupid abbreviation,that's just Raspberry Pie. I probably shouldn't abbreviate that.
I don't know if that's their official one. I think it might just be Pie, but yeah, R Pi Raspberry Pie. Sorry about that. Uh, once you have so much stuff,you gotta do acronymsand abbreviations so no one knows what you're talking about. I mean, it makes it fun for everybody.
Sorry about that. Uh, so if I wanna run VUS Light locally,which runs great on your laptopor on any of these Unix, it's pip install. Uh, obviously you want, uh, a newer version of, uh,Python three on there. And that's all you gotta do. Install.
Now it's running, uh, and available locally. I could run it in Docker. Obviously, I gotta use either the Docker fileor a Docker compose,or I could also run it in Mini Cube or something else. I can also run an Edge server. There are edge devices, which are even have more storageand more CPU and more ram.
And you can run the standalone version thereor the Docker composed version,or even a tiny cluster with a couple of them. What's nice is, uh, it doesn't matter, you saw we just haveto change a couple variables,and now you talk to that one pretty easy. Uh, it's the cloud. I add a token. This is the difference on the, your Python source code.
You know, do I point to a local database file?Do I point to a local server?Do I point to the server and pass in a token?Uh, it depends what you're doing. Pretty straightforward. Now, what's nice too is when you're dealing with the Edge,obviously Python is first class citizen. A lot of people write their apps there, but other appsand languages are supportedand are pretty common on some of these devices. Like no js, Java, surprisingly is on a lot of 'em,and it works pretty well.
You're coming from the Java space. Let me know, I've got like 20 years of Javaand there's, there's use cases for,uh, VIS and Java together. Golan obviously native, uh, cloud languageGo works really well in all these environments. You, you wanna just do it with Curl. There's restful APIs, you don't have to write serious code.
Uh, we're gonna have a, uh, webinar on. net C sharp version,which is looking pretty interesting. We also have Spark and Kafka and Ruby,and a lot of, uh, user developed interfacesand languages out there. So if you're interested, there's a lot of options. It doesn't just have to be Python.
So obviously Python is pretty easy to do. Uh, as we mentioned, lots of different use cases. Uh, anomaly detection at the edge is pretty cool. Uh, being able to search, uh, audio. So if you, you know, use that for security, you know,have your own device, answerwhat you want and just have it locally.
It doesn't have to go to the cloud to, uh, sharethat you asked for, uh, where the Pepsi isor where's my novis, uh, doll,you know, you don't have to do that. Facial recognition, again, dealing with autonomous thingsand, uh, robots, again, being ableto be low latency, offline, secure,localized storage, pretty awesome. And then once I get it to the cloud,you know, the world is your oyster. Wherever cloud you wanna run on whatever kindof fancy hardware you may have. We've done a ton of different embedding models from, uh,some open source, some paid like open ai,but you know, whichever one makes sense for your use caseand your data, you don't have to use a paid one.
Uh, there's a bunch of good ones in the open sourcethat work really well,and there's a ton of awesome LLMs out there. And we work with all those frameworks. You don't have to worry about, well,if I'm using Lang chain, can I still use vis?Yes, you can. Well, you run that at the edge. Well, you could, especially on an nvidia.
So I got data, I'm doing it all locally,and then once a week, once an hour, once a minute,I don't know, once a day or once,or maybe when you, you have it for an event,say I put it at the Olympics when it's done, I want to putthat in the cloud and put all my data from my 10,000, uh,remote devices together. While I can very easily dump that data from VUS Lightand import it right into the cloud. We mentioned that dual ingest, the old,uh, Hadoop Spark way. I could just make things push directly to the cloud. Again, you do have to have a network.
They are pretty prevalent. But when you're doing, uh, distributed programmingwith Edge devices, you gotta think about that. We could all use the triedand true, uh, methods for, uh, messaging. Uh, I can message the data through Kafka or Pulsaror something, uh, IOT specific like MQTTor some other protocol. If I want to send this out somewhere.
Uh, we support importing Parquet files if you'refrom the big data space. So you could send this to a regular Kafka Pulsar app,have it, dump it to a big Parquet file,use something like NI NiFi,and then we could just import that into, uh, the cloud. You know, we could push this in structured data to an,uh, object storage. And then again, send the other part of the messagethrough something like, uh, Kafka or Pulsaror just send the whole thing right to, uh, vis depends onwhat security performance. Oh, smart.
Yeah, it's, uh, blip. Uh, I'll show you the, uh, source code there. Somebody knows this stuff. Yep. Uh, sure.
Um, that I don't know. Um, we'll have to take a look at that one. Um, so Viss Light exports command line toolcomes with the install. You just point through database, point towhere you wanna put it and pick your collection. Uh, you could have multiple collections.
So I, I would probably do one database per collectionbecause there is, you know, a lock on thisas it is a regular little file. It's not the real vis. They could do all the fun. Concurrency, something to think about. One database, one collection, one app.
If you're doing these, especially on these small devices,I mean, how many are you gonna do there?But yeah, just export thatand you could install it somewhere else. Uh, I've got the source code for all of these. Uh, I will show you that along with articles describinghow we did it, what algorithms, uh, examples,uh, you know, all the libraries you need, you know,the full packing list there, howto get started, example runs. So it is, it is not, uh, too hard there. This one is from the older Nvidia device.
Again, I got this, I didn't write that article yet. I'm feeling bad, but I wish we had someextra days in this week. Maybe, maybe somehow I'll push it through. Uh, I want to do some more advanced use cases for that. Um, for that, uh, let's get to that source code.
This was the one for the, uh, the, uh,that we did on the Raspberry Pie. We did two different ones on the Raspberry Pie. This one I did just recently. And this will come out in an article very shortly. Uh, this is for the Pose estimation.
Uh, we have another one here. Uh, this was the original, uh, AI kit one I did. And this is for just detection. We've got all those source code for that one,and that is a giant corn. I won't tell you where I saw that Giant corn, uh,Xavier Edge.
This is I think the one you were asking about. And we've got the source code here. As you could see, it's the blip processor that, uh,you got that one right?And we have blip down here. So I capture, uh, image off my camera,run it through blip. I'm using, uh, hugging face there for that.
And I think it's the Salesforce slip version. And that runs pretty good. Grab the caption, put that into vis,and post that into Slack. The, uh, slack library for Python. Really nice, if you haven't seen it before.
Oh, dual ingest is a technique that was,it's not optimal,but it, it's a way, if you needto populate two databases or two things at the sameor similar time is exactly what it says. Dual. I send data here, send it here. That could be multi-threaded. It could be multi-process or it could be serially.
But what it is, is I wanna have a copy locallyand I wanna copy somewhere else. Now, easiest way to do that is just to do that. Connect to this database, insert it here, connectto this database, insert it there. Uh, inserts are really fast. So unless you're doing thirty, sixty, a hundred twentyframes a second, and you're pushing a lot of data throughthat, that is an easy way to populate 'em.
Both, obviously, you could also have an app that checksand sends it later. You could also write the, uh, logical J that would goto be an insert and store that and have a app do it. There's a lot of distributed programming techniquesto handle, sending things to multiple places. Viss doesn't have Auto Federation yet. Uh, I don't know if that's somethingthat they think about in the unstructured data world comingfrom the structured data world.
You know, people like to do that. That might be a thing. Or maybe you just send it to the cloud. Again, nothing that's not an advanced technique. That's pretty, that's, uh, the simplest wayto do something, to send it both places.
You know, if I, if I have to send, uh, it's, you know,send a carbon copy somewhere. Uh, if you liked what we're doing, please give Vis a star. You know, the more people in the community taking alook, trying things out. You could be a commander by just doing documentationand documentation's really important. Doing examples, looking for simple blogs, typos.
So definitely check out our source code. We are on Discord. There is a lot, lot of cool content in there. There's also an AI bot to auto answer questions,and most of the engineers are on there. Maybe not every day.
There's always at least one engineer on. So if you've got a questionor just comments, discord is a great way to do that. Uh, a lot of stuff going on there. We added a Tim,Sorry, sorry to interrupt. Run outta time.
Uh, you have a question that, um,someone wants to answer, like unmuted, so I'm going to, uh,oh, allow him to ask his question. Sure. Uhand, I'm gonna, uh,unmute you if you'd like to ask your question. Now. Um, he raised his hand, soYeah, yeah.
Or you can type it if that's easier for you. Um, yeah, I know you raised your hand some time ago. Oh, I hope you didn't lose your questionunless it was really hard. Then I would just say the answer is 42,Or if it was already answered, maybe,Oh, that could be, that could be you, you say enough stuffand randomly it's the, the right answer. Uh,There's, okay, well if it comes up, uh,please put it into the chat.
Um, and we'll get that answered. Yeah, and if, if you don't get it now, put it in Discord. Send it to me in, uh, Twitter and LinkedIn. Uh, come to the meetup if you're in New York or Californiaor Berlin, uh, or just, uh, email me. I'm always looking for cool stuff.
Or put it in, uh, stack Overflow where we tryto monitor all the vis channels, looking for, uh,questions and, uh, comments. Uh, we've got cool things you could try online. Uh, if you want to try your own demos there withoutinstalling anything, we've got the cool OSS chat,which is an example of using vis, uh, to chatwith different, uh, open source documentation. And it is a pretty cool way to do that. Nice little example.
Uh, if you want to try things out, we could try, um,Zillow cloud for free. If you don't want to install anything,you don't even wanna install the Python library,you know, up to you. All the information is there. If you want to see more of the demo,we got a couple of minutes. Uh, otherwise, if you got more questions, uh,I'm not seeing any yet.
But, uh, if you didn't fall off,and if you're still on, you want to ask that questionand, uh, q and aor, uh, chat, let you know. Uh, let's, you have about four more minutesif you'd like to share something. Okay. Tim?Yeah. Okay.
Let's, uh,yeah, we got the Discord. Yeah. Uh, all the slides will be shared. All the links to the source code are shared. Uh, pretty straightforward.
What's nice is with, uh, with, uh,Viss, regardless of where I captured that dataor where I run the database, I can very easily connect to itand, you know, connect to it through a notebook. Uh, like this example, I'm connecting to the cloud for, uh,travel advisories. This one's pretty fun'cause we're using, uh, BM two five justto show you different ways you can, uh, you can connectwith sparse, uh, inverted indexes. I mean, we, we support so many different ones. Uh, it depends on your use case, which makes sense for you.
Uh, one thing that's cooland add to which I liked is you saw this search here. Uh, what's nice is, remember we saiddifferent consistencies?Well, if you're not concerned that the data's done,you could put eventually in,like if you're inserting things like crazy,which maybe we can insert a couple more things. Uh, which one are we on? That is local?So let's run the server. Remember we said, what are those different runs?That one's looking at, uh,lemme move the camera a little bit, see ifthat gives it a little different thing. So that'll be here, like while it's running, you know,you could pick, uh, strongwhere you don't want it available till it's fully there.
All those sort of things. You could change how you groupby things, uh, would wanna group by doing current timeand, you know, get your results. If I want to export this, I've got a CSV file,which I can now, uh, use for whatever I want there. Uh, you could filter by whatever you need to. Again, pretty cool.
If you wanna just look at the raw data,pretty easy just to browse through it. Again, probably don't wanna look through the vector. It's, that's a lot of numbers. It's kind of like looking at the matrix. Maybe you see stuff if you look at it enough times.
But, uh, I don't, um, uh, for this one,this was the one from the travel advisoriesat which we take an RSS feed, process itand get it in here with, uh, a fast index. Again, searching stuff here is, is is very fast and easy. I mean, regardless of where we're running it, uh,what other source could we have?The pose estimation is new. Uh, this is one of the examples that come with the, uh,raspberry Pi five AI kit. And, uh, I just, uh, co-opted it for my own purposes.
And, uh, every couple of rows I insert it. 'cause otherwise I had thousands of rows of, you know,my head not moving much, which is not very exciting,especially when I'm sending all those messages to,uh, to Slack. Uh, again, you put whatever you want in your app,but I, I like to output the slack in these distributed, uh,event systems just'cause very easy for me to know what's going on. Good for debugging. Also good to, uh, be ableto share that information.
Uh, you know, and this is an open Slack groups,if you wanna join, you could joinand you could see my weird messagesthat get auto, uh, posted there. And, uh, there's some of them connected to, uh, AI chat,you know, all those sort of things. Uh, if anyone is interested in coming to, uh,my meetups coming upor the other meetups, we've got, uh, a bunchof them coming up in the next, uh, couple of weeks. We got one in San Francisco at GitHub. I'm still trying to figure out, uh, where mine's gonna be.
We're we'll be in Manhattan. We're trying to find a big enough space. 'cause uh, last week we had, uh,260 people register. I know a lot of them were gonna be virtual,but we wanna make sure everyone has a seat. Everyone gets, uh, some food.
And we've got a bunch of speakers for this one. I'm doing a short talk. We've got an interesting guy who's developed a, uh,reporting search tool on top of Vis as alongwith some other databases,which lets you do sql, which is pretty cool. We got a principal engineer from Nvidia. Got my friend David coming in from stream native doing, uh,gen AI plus streaming.
We got Jacob Marks from, uh, voxel 51 coming on, doing some,uh, interesting stuff. So it's gonna be a packed agenda. So definitely come out to the meetups. Definitely give it a try in the environment. Uh, I hope we answered that question before.
If we didn't, uh, we'll definitely, uh, reach outand put that in the, uh, follow-ups. Uh, thank you for coming to the, uh, webinar. Hope you had a good time. I thinkwe're probably out of time here. Yeah, yeah.
So we're at the top of the hour. So thank you so much everyone for joining. If you have any follow up questions, um, Tim's QR codeto his LinkedIn is on the screen. Uh, connect with him on either LinkedIn or Xand he'd be happy to answer your questionsand join our mil mil discord if you have, um,mil related questions. Okay.
Thank you all, uh, for joiningand we'll see you at the next one. ByeBye. Thanks for coming.
Meet the Speaker
Join the session for live Q&A with the speaker
Tim Spann
Principal Developer Advocate
Tim Spann is a Principal Developer Advocate at Zilliz.