- Events
Building a GenAI App End-to-End with Ruby
Webinar
Building a GenAI App End-to-End with Ruby
Join the Webinar
Loading...
About the Session
The Ruby community continues to adapt to the constantly evolving AI/ML needs by building tools and methodologies suitable for the Ruby stack.
Join this live session with Andrei Bondarev of Source Labs who authored Lanchain.rb, an “original Langchain”-inspired open source library for building LLM-powered applications. It supports use-cases such as semantic search, chat bots, Q&A, prompt management, and experimental agents. Combined with the Ruby SDK for Milvus you can build a GenAI app end-to-end with Ruby. Come learn how!
You’ll learn:
- What vector search is
- Use-cases that Milvus helps solve
- Building AI-driven application with Ruby
- Ruby API client for Milvus
- Langchain.rb, an orchestration tool for building LLM-powered applications
Today I am pleased to introduce today's session, building a Gen AI app,end-to-end with Ruby and our guest speaker, Andre Bond. Andre has been a software engineering professional for almost 13 years,among many others. He's been fortunate to make his impact at Spree Commerce,WeddingWire, fiscal note, national Public Radio, and u s A today. He currently runs a software development firm and serve as architect engineeringmanager and fractional C T o, um, on client projects. In his free time,he enjoys playing tennis and going on long runs while listening to podcasts.
Uh,Andre is also joined by my colleague Yuin Tang,a developer advocate here at Cylis. Welcome Andre and Yugen. ThankYou for that one. Welcome, Emily. Um, so hello everybody.
Thanks for coming on today, we're gonna be talking about generative AI and Ruby. And so, uh, first I'm gonna let Andre introduce himself. Hey, how are you? I'm Andre. Um, Ruby has been my tool of choice for, uh,over a decade. Um, and I've been actively thinking, um,that we ought to have, uh, a, uh,a AI solution, uh, to address the, uh, growing amount of needs in that space.
Um, and I'm here to, uh, talk about that today. Thank you. Yep. Thank you Andre. And also that little QR code, if you have your phone,you can pull it out.
I'm gonna give you about 30 seconds here to scan the QR code,and that QR code will send you to Andre's LinkedIn and you can connect with himon there and ask him, uh, questions about Ruby?Okay. Oh, so my name is Jun. Uh,I'm a developer advocate at Zillows. Um,I've worked at companies like I B M and Amazon. I've worked on auto ML stuff.
I've published research papers and I'm big into machine learning things. And so I work on a lot of generative AI and how you can build l l m apps, um,using, uh, the new C B P stack using a chat G B T, which is a, uh,or a chat G B T, like l l m and a vector database and prompts code. Uh,and just like Andre,I also have a QR code there on the right side of yourscreen, and you can scan that and you can find me on LinkedIn. Okay, so let's get today's session started. Uh, oh, wait,one more slide.
Okay, one more slide when we're talking about Zillow. Uh,Zillow is a Vector database. Uh, zills is an unstructured data platform. Viss is our open source, uh, vector database,and you can see our LinkedIn and our Twitter as well as where you can go jointhe Viss Slack and check out the viss GitHub. Um, yeah,so Mils is the open source factor database.
Zillow is the cloud managed version,um, which we also have a free tier for that you should check out. Okay,so let's cover today's, uh, agenda Today. We're gonna start with introduction to generative ai. So I'm gonna talk a little bit about, uh,neural networks and vector and vettings and vector vector search, and how,uh, things are kind of looking right now in generative ai. Then we're gonna go into generative AI with Ruby, where Andre's gonna take over,and he's gonna talk about, uh, Ruby and generative ai and why these two,you know, why, why it matters.
Uh, then we'll talk about Viss with Ruby. So Andre made the, um, Ruby s d K for vis, and, uh,we'll cover a little bit about the Mils architecture. I'll cover a little bit about the Mils architecture,and then Andre will tell you about, um, the Viss, S D K, and then,uh, lane chain RB or Lang chain for Ruby is Andre's, uh,other project, which is Lang Chain, uh, but for Ruby. And he'll talk to you about that and show you some really cool, um, demos. Okay, so let's get started with this introduction to, uh, generative ai.
Okay,so we're gonna start by just thinking about the amount of data that's out there. So basically, every corporation runs on data, everyone runs on data,and most of that data is unstructured data. So,I d c estimates that there's gonna be 175 zetabytes of data globally with 80% ofthat being unstructured, and 90% of that unstructured data never being touched. And unstructured data comes in many forms, texts, images,videos, PDFs, CSVs, all of these things are counted as unstructured data. And the reason is because unstructured data is basically any data that doesn'tfit your predefined data model.
And so how do you work with unstructured data?You gotta use vector embeddings. The way that you are able to actually interact with and work with yourunstructured data is you take your documents, um,you put it into a embedding model. And so that work,the way that works is you choose a, uh, fit corresponding deep learning models. So if you're working with images, you probably want resin at 50. If you're working with, uh, sentences, you probably want sentence transformers.
And then you cut off the last layer,you cut off the last layer of your model so you can get the vector embedding. The vector embedding is the output from the second to last layer, right?Because the last layer is usually some sort of classification and prediction orsomething like that. And so that output gives you vector,and you put those vectors into a vector database such as Novus or, uh, Zillow. And you are able to query and interact with your un unstructured data once it'sin a vector database. Um,so let's take a little bit look at what are these vector embeddings, right?I've just said, you know, how do you work with unstructured data?You put them into vector embeddings, but what is an actual vector embedding? Uh,here I'm gonna show you a toy example of a vector embedding.
So these are four examples of, um, two dimensional vectors,and you will never work with two dimensional vectors in real life,but this is a illustrative example, right? So let's start by looking at queen. Um, so queen here is represented by 0. 3, comma 0. 9,and king is represented by 0. 5, comma 0.
And woman is represented by 0. 3, comma 0. 4,and man is represented by 0. 5 comma 0.
What I wanna draw your attention to here is that woman and queen start with thesame, uh, you know, X value, I guess,and king and man start with the same X value. And what this tells you is that along this axis, along the X axis,these words are similar and these words are similar. Um,and you can also see the difference here along the y axis between woman and manis 0. 2 as the same between king, queen and king.
So what vector embeddings allow us to do is they allow us to do math on thesekinds of words. So we'll start with, uh, this example here. If you start with queen, you know, 0. 3 comma, 0. 9, and you subtract a woman,then you are left with zero here.
And, uh, comma 0. 5. So with zero comma 0. 5, you add, man, then you get 0. 5 comma points,which is the word king.
So the whole idea behind this slide is just to illustrate the idea that you cando math on words using vector embeddings. So things are generated, like I said before,they're generated from the second to last layer. So this is what a neural network kind of looks like. Um, tally, right?You input some data and then it goes through a bunch of hidden layers,and there's a ton of calculations being done. And typically there's an output layer,and the output layer is some sort of classification or prediction or regressionor something like that.
And, um, what a vector embedding is,is all the information that goes to classifying what that input,uh, should output, right? So that's how you know that this is this semantic, uh,meaning behind, um, this inputbecause it's, it's,it's gonna be interpreted by the output to give you some sort of classification. Uh, this is just a screenshot from Zillow to give you an idea of what, um,vector embeddings are and what they look like, right?So the important thing here is to look at this title vector here, right?You see how these vectors are all like, you know,these float values and there's like a lot of them and they can be negative, but,um, or positive. Um, so yeah, vector embeddings are basically this long,long string of numbers. Uh,so where can vector databases help some use cases,some like real life use cases are, um, listed here. Um, basically, you know,similarity search is really the, uh,the abstraction that goes across all of these images, videos, audio texts,d n a, you know, whether or not something is out of place, right?Anomaly detection is really just similarity search with a different point ofview.
Um, so yeah, these are some things that vector databases,spec databases are helpful for. And the way that similarity search works is essentially you take yourunstructured data and you transform it into some sort of vectors. And then once you transform it into a vector,you save that vector embedding into your vector database, such as vis. And when you need to, uh, query, you perform a query,you take that unstructured data that you had before and you put it into a vectorembedding, uh, into the same vector embedding model. Uh,typically the same vectors embeddings model,you can actually use a different one,but you'll have slightly different semantic meanings.
Um, and, uh,one thing, if you do wanna use a different one, I don't suggest it,but if you do wanna use a different one,you have to make sure that it has the same number of dimensions. Once you have your new vector vetting,you query against the vector database and it performs an approximate nearestneighbor search, and you get some results. And that's kind of how you find,you know, what are the most similar vectors to my, uh, input?What are the most similar data to my input?So an example of a generative AI use case of vector databases andLLMs and, you know, uh, this new C V P stack, um,the example that we're gonna look at here is saying like, Hey,like let's say you, uh,are a company or you work at a company and you have hundreds of thousands ofmaybe millions of pages of documentation that your, uh, users,that your customers, your staff, your, uh, you know,software engineers that use this kind of, uh,data that they need to have this documentation. Um,you can either have people search it, which takes a long, long time. Um,this was definitely what my experience at Amazon was like,or you can create an internal chat bot using the C B P stack, right? Chat,G B T,like L l m A vector database and some sort of prompt and you can talk to your,um, data that way.
So that's a really good use case of generative AI and veteran databases. And now we're gonna talk about generative AI with Ruby,and I'm gonna hand it off to Andre to, um, share your screen. Thank you. Oh, alright. So let's talk about generative AI with Ruby.
Um, so I run a small software development company called Source Labs. Uh,so we help, uh, our clients bring amazing software products to market. Um, and we're obviously doing a lot of, uh, AI work now. So why Ruby? Um,obviously Python is kind of a leader in the space, um, because it has,it's so popular in the academic setting, in the research setting. Um,Ruby is very similar.
It's also, uh,built in seed has a lot of kind of similar issues, um,same problems with concurrency, very similar syntax. Um,and, um, Ruby and Rails are still very popular. There's still a lot of, uh,there's a lot of, uh,brand new products and existing products that are built in Ruby, um, uh,on Ruby on Rails. Um, and, um, subjectively,I think Rails is, uh, one of the best ways to, to build web services. Um,I find that the community has this healthy amount of pragmatism when it comes tobuilding, uh, tools.
Um, they tend to not reinvent the wheel. Um, and then it's also very conducive to the monolith application structure,structure architecture, um, in this kind of environment of, um,economic stagnation or contraction, depending on who you ask, uh,companies are looking at their expenses and they're looking to cut costs,um, in, uh, many different areas. And simplifying your application architecture is one of them. Um,so as opposed to running, uh, uh,running an array of microservices and that brings, um,that brings your cost up and it's, uh, a lot more complex, um, as opposed to,um, running a single monolith, for example. So also in this, uh,in this, uh, diagram below that I borrowed from, um,a very good blog post called The Rise of, uh, of an AI engineer,you could see that a lot of the traditional full stack engineers on the righthand side are getting a little bit closer to this a p i boundary.
And now they, um,need to be much more familiar with ML ai,L l m concepts such as chains, agents,and the associated tooling and the infrastructure. Um, so,uh,and but the language here really kind of doesn't matter whether it's Python orRuby. Um, so, uh, we're trying to build a set of, uh,tools in Ruby so that, uh, the EC engineers can, can, um,use these, um, APIs. And, uh, let's talk about VIS with Ruby. Um, and now Egen is gonna explain us what, what VISS actually is.
Yeah. Um, great explanation for the,the AI engineer stuff and why people need to understand what's going on with L LM Ops entree. Um, so Viss and Ruby is one tool that you can use,uh, that's in Ruby that will help you build these l l m maps. Um,the architecture of VUS is cloud native, distributed system native,so it's pretty complex, um, but I'm gonna try to cover it in a high level. And if you have, uh, more questions, please, uh,feel free to drop 'em in the chat and we'll answer them to the best of ourabilities.
Uh, so with Viss, you start with, um, you know,hitting some sort of S D K such as the Ruby s d K,and from there we kind of like route your queries, um,into the worker nodes. And so this is really the, uh,there's really a couple important things that I wanna point out here. And so one is that you see these three kind of nodes under worker nodes. One is query node, one is data node, and one is index node. And the reason why we have these three different nodes is because actuallyquerying,ingesting data and indexing are three different separations ofconcerns when it comes to working with a vector database or any database,really.
Um,and so what this allows us to do is this allows us to dynamically scaledepending on the kind of, um, the kind of, uh, uh,workload that you have, the kind of workload that you need. Um,and another thing that is really, really cool, and this one, like when I was,when I, when I was, when this was first explained to me, this like blew my mind. Um,VISS puts VEC creates vector indexes over 512 megabyte chunksor segments, uh, sorry, segments is what we call them. Um,and then when at query time,it is able to parallel query all of thesesegments and return a response based on that parallel query. And so you can imagine that as you grow your data size,so for example, at a hundred, uh, gigabytes,you're gonna be doing a lot more querying if you're,you're gonna be doing a lot more computation if you're gonna go through all ahundred gigabytes of this, uh, in,in sequence as opposed to in parallel searching 512 megabytessegments and then, uh, coagulate or getting everything together at the end.
Um, so that's one. This are like two really cool things, um,about novis that make it really scalable and, uh, makes it really unique. Um,so yeah, Andre, uh, you know,I'm gonna hand it off to you to talk about the Ruby aspect of the Ruby s d K. Thank you. So if you decide to try out vus, um,we think you should, um, we have a Ruby a p i client,uh, and you can find it, um, at that link right there.
Um,it is one of the popular options for, uh, uh, building a Vector search. Um, it's a convenient wrapper and Ruby on top of VUSs. Um,it's part of the length chain or B stack, um, and it supports,um, managing your vus, uh, database. So you can create collections indices, uh, add or remove data,search query and managing, uh, and manage partitions. Uh, so let's,let's jump into what, uh, link chain RB is.
Uh, so it's inspired by the original, uh,Python slash TypeScript implementation. Um,again, I felt like we ought to have a solution in, uh, in the Ruby world, and,and, and I kind of dove in and starting building this, uh,link chain interview project. So it's,it's an orchestrational layer for building l l m based applications. Uh,it's for stringing multiple, uh, types of systems together. So vector search databases, uh, LLMs, um,uh, your various data stores, uh, and kind of auxiliary services around that.
Uh, some of the use cases that we target is semantic search, vector search, uh,QA over documents, chat bots, and, and the, uh, experimental agents. Uh, and it's also, uh, it presents the developer with the best practices. Uh,it encapsulates, uh,kind of the best approach for building applications with LLMs. Some of the, uh,vanity metrics here, um, it's only about, uh, three months old. Uh,there's about, uh, 480 stars, 30 contributors and growing,and we also have a Discord server.
Um, some of the features, uh,we have within link chain rrb is prompt management. So being able to create,load and save, uh, prompt templates, context length validation. So every L o M has its own kind of, uh, context length, uh,window limit, um, data chunking. Uh, so anytime you're indexing data into vector search database, uh,you need to make sure that, uh, your chunks make sense. Uh, semantically,uh, we've got conversation memory, so persisting and ongoing, uh,conversation with an L L M to memory,and then kind of picking it up and revisiting it later.
Um,so you could see that link N or B lets you useany sort of vector search database with, um, any, uh, matching,uh, L l M. So, for example,you could take Novus and you could use philanthropic for embeddings or GooglePalm for embeddings or open ai, of course. Um, some of the advantages behind this approach is that it offers, uh,a common, uh, do domain specific language or common set of APIs. So because of that, you have interoperability. So if you wanna switch out from open AI to, uh,uh,to Google Palm or Claw two from Anthropicor, um, you know, any, any, uh, any other emergent, uh,lms, uh, or using an open source l l m use, uh, LAMA two, for example,you can.
Um, so because of that, there's a reduced vendor lockin. Um,you've got optionality because of that. Um, and again, we,uh, try to enforce the B best practices,which we try to be a little bit opinionated with link Gen B, so that, um,uh, so you don't have to make those, uh, minor,minor decisions at every, uh, at every fork along the way. Um,so let's dive into some code. So how would you execute vector search with,let's say, VUS as Azure Vector Search database and Google Palm as a, as your,uh, foundational model? Uh, so you would instantiate your l o M here, uh,just merely pass it in a p i key, uh,you would create a client, uh, using the vis, uh, vis, uh,Ruby Gem, uh, pass it, the, the U R L where your, uh,VU instance is running name the index pass, uh,pass the l l m in, uh, instance.
Um,and now you're ready to start, uh, indexing data. So you could either call, add,text, and just pass an array of different texts and corresponding IDs,uh, or you could send it files. That link chain RB will actually load, parse, uh,and index into vis. Um,if you're familiar with active record, which is, um, uh,an O R M in the rails world, um,you could accomplish the same thing in, in a lot fewer lines of code. So in your active record model.
So for example, this one is called product, uh,on top of a products table database table, um,you would declare your vector search, and, uh, again, in this case,we're using VIS here. Um,and then you would set up hooks so that whenever that record in the database isbeing operated on, uh, for example, MySQL or Postgres, uh, we would alsogenerate embeddings and update, uh, that record in viss. So you could see that we will upsert to our vector search, um, in, in this case,viss. Um, and once you've done that, once you have some data in your,in your VISS database, um, uh,you can start executing similarity searches. Um,so you could do a simple, um,client dot similarity search, passing your query,or you could do QA style querying based on the question passed in.
So let's take a look at the demo. So you can see that I'm instantiating the vis client. I'm creating the default schema, creating the default index. Um,I'm loading that default schema into memory. Um,and I should mention in, uh, in this video I have, uh, vis, uh,running, uh, in, in Docker in a separate tab locally.
Uh, so I'm actually pointing it to, uh,import a benefits brochure, um, your company, for example,your company's benefits brochure. Andthen we're just gonna go ahead and add that to, to Novus. Uh,and I'm just gonna scroll ahead a little bit. And now that that data has been indexed, uh, that, uh, benefits brochure,P D F has been, uh, loaded, parsed, indexed into Novus,I can now do q and a style, uh,question and answering on top of my corporate data. So for example,you could see I'm asking what is the vacation policy?How much time can I take off?The company's vacation policy allows employees to take any reasonable amount oftime off with pay.
Um,what are all the benefits offered? The benefits offered are paid time off,sick leave, paid, paid holidays, et cetera. Tell me about the 4 0 1 K benefits of the company. Uh, and you've got options such as Roth, uh,or standard 4 0 1 K, yeah, tradi,traditional and Roth 4 0 1 K, what are all, whatever the official, uh,observed holidays and you get an answer back. So, um, let's see. So what we were just doing in that demo, we were doing retrieval,augmented generation, um, and,uh, what it is,is retrieving data from outside the foundation model and augmentinga prompt by adding a retrieve data into the context.
So, um,whenever a user's question would come in, uh,we generate an embedding vector from that user's question. Uh,we run a similarity search against the data in our vector search db. Um, again, we were using vus. Um, and then we take those matches,we put them as a context into the prompt, and then prompt the l l m to generate,uh, a coherent answer. And this is, uh,this is the actual this prompt that we were using.
Uh,so let's talk about agents. So agents are autonomous, uh,general purpose L l M power programs. Um,there's tons of emergent research, uh, such as chain of thought,tree of thought, uh, uh, reasoning plus Act, react,et cetera. Um,and they can be used to automate workflows or executemulti-step, uh, tasks. They work better with powerful lms.
So for example, Claude OpenAI, uh, Google Bard,um, I haven't actually tried it with ATU yet. Um,and they're usually built by care with carefully engineered prompts. Um, there is a need for, um, uh,for careful air handling and, and constraints for, for resiliency. Um, and they can also use tools which are basically, um, uh,a p i wrappers. They're kind of similar to open AI plugins.
So some of the tools we have available,and it's really easy to extend and add new ones are, are the calculator,the Wikipedia, a p i, the Google search, a p i, the web a p i, um,you can also interface with your own database. Um,so I'll give another, uh, quick demo here. So again, we're using Han RB here in, in a, uh, local,uh, ripple session. So we're gonna instantiate the weather tool, um,and we're using this outside service. We're using the Ruby Code interpreter tool.
Uh,we're using a Google search tool, and again, we're using an outside service to,to do that again, uh, the calculator tool,we're gonna use the open AI L l M in this case. And, and, uh, we're gonna instantiate our React agent. Uh, so we're passing it, the l l m, uh, in this case,the open AI and the list of tools. So we've got the weather tool,the Google search tool, the Ruby Code interpreter tool, and the calculator tool. And the first thing we're gonna ask is we're gonna task it and say,find the current weather in Boston,Massachusetts and Washington DC and take an average.
Um,and what it does is it sends the originalprompt to the OpenAI. OpenAI says, okay, well,I need to go and find the current weather in Boston, Massachusetts right now. So, um, we recognize that we execute the,the weather, uh, a p i, we fetched the, the, the current weather in Boston,Massachusetts. We send it back to the L l M, uh, the l l M says, okay,well now I need to know the weather in Washington dc. So we repeat that.
We send a, uh, an a p I called, uh, to the weather, a p i,we fetch the current weather in Washington dc And then lastly, the L l M says,okay, well now we need to use the calculator, uh, tool, um,with these inputs, the, the current weather in Boston,the current weather in Washington, dc, um, and divide, uh, divide by two. And you can see it comes back with a final answer, which is the current, uh,the average current weather, I'm sorry, uh,the average current temperature in Boston,Massachusetts and Washington DC is 84 degrees Fahrenheit. So the next question we're gonna ask it is we're gonna say, what is the current,uh, Rubal U Ss D exchange rate?So remember that we're using, uh, OpenAI and, um,uh, OpenAI has only been trained to, uh,to data up to, uh, i, I believe mid 2021. So it, it has no clue about what the current, uh, exchange rates are. So in order to fulfill, uh, or answer this question,it needs to do a Google search, uh,and we need to find what the current exchange rate is.
So when we send this question to the L L M, it says, okay, well,I need to go and do a Google search for this. Uh,we execute that a p I call on the M'S behalf,send back the result to the L L M, and it's able to synthesize an answer,generate an answer. Um, and lastly,we ask it to, uh,use a Ruby program to output the sum of Fibonacci of this Fibonacci sequencefor one through 1000, and not to,and we instructed not to define any Ruby methods. So it comes back with this code. Uh,this is a range, um,and we use the Ruby Code interpreter tool to execute the Rubycode and send it back the answer, and it's able to put it in,in natural language.
Uh,the next agent I wanna show you is the SQL Query agent. Uh, and in this case, we're gonna use a database, which is running locally,which is a, uh, Postgres instance. Uh,and it's a simple database, and it, it is a e-learning platform. So we have students, uh, which is in the user's table. Uh, we have courses,which is in the courses table.
And also whenever students, uh, complete courses,they receive certificates, which is in the certificates database table. Um, so we in, so we instantiate the database tool. Again, in this case,we're using the open ai L l m, we're instantiating the agent,uh, the SQL Query agent. We're passing the, uh,l l m instance and the database tool,and we ask it, uh,the first question that we're gonna, uh, the first task that we're gonna,um, impose to the agent, uh, or we're we're gonna ask the agent,is we're gonna ask it how many users are there?And the l l m will convert, um,this question, natural language question to a corresponding SQL query. So it will say that the proper query to answer this question is select countstar from users.
Uh,and then we will use the database tool to execute the query and then send backthe answer, uh, to the L L M. Um, and then the fin. And,and then the l l m generates a final answer, which is there are,uh, 2,589 users, which is correct. So the next question that we are gonnaask it is how many users have completed all four courses?Uh, and the L l M looks at that question and says, well,in order to answer this, given the database schema,um, this is the query that you need to run. So select Count Star from users,inner joiner certificates where certificates complete equals true.
We take a query, we execute it on the l l m's behalf, send it back the answer,uh,and the l l m comes back with 160 users have completed all fourcourses. Um,and this, uh, concludes the length chain RV part. Uh, so, um,this is our discord. Um,we're very open to any kind of feedback, feedback, um,any kind of new ideas how to improve the tools. Um,and we would just love to hear what everyone else, uh, is doing in the space,uh, especially if you're using Ruby.
Uh, so please, um, if you get a chance,please, please do join in. Um,and that concludes, uh, uh, our session. Thanks for such a, a great, um, overview of,of Lang Chain and Ruby, and what you can do with the,the Lang chain Ruby client already. Um,I'm gonna let other people ask questions, but if it doesn't look like, uh,we have any open questions at the moment, um, I, someone I saw that,uh, Schenker asked about can we use Spring Boot j p a for connecting to Vis,similar to Ruby? Um, I think you can, we have a Java client,so I dropped the Java client in there. Um, and if that helps you, please,you know, let us know.
Uh, if that's not what you're looking for, um,you know, feel free to feel free to ask. Um, but I, I,I already find your, your lane chain Ruby demo to be moresmoother than my actual Python lane chain experience experience. Um, so I, I kind of wanna know, like, you know, uh,what is your kind of take on how you wanna see Lang Chain Ruby evolve, uh,evolve?Um, that's a very good question. Um, you know, I think, I think we're,I think we're trying to avoid building tools for, uh, for the tool's sake,um, and, and have everything rooted in real world use cases. Um, so, uh, we're constantly trying to solicit,solicit feedback here.
Um, you know, you can,you could kind of, um, you can read some anecdotal data on,on Hacker News and Reddit and Twitter, other places that,um, people do struggle. Um, in the Python world,people do struggle with some of these frameworks, um,and understanding and understanding the concepts. Um, I think,I think Ruby, like I mentioned, I've mentioned, uh,kind of pragmatic driven development, uh, sure, yeah. Before, um,and I think the community really has a chance here to bring those, uh,bring those values into AI development as well. Um, and, and avoid,avoid reinventing the wheelForSure, man.
Um, I, we we're, we're also, we're also trying to build a much, um,much tighter integration with Rails. Okay. So that it's,I don't wanna say plug and play, but it's as close as as as close as possible. Yeah. So because, um, because I think,um, I think definitely this year, um, with,with this AI wave, um,all of the product managers are coming to their engineering leads and, and, uh,engineering leadership and are, and are saying, okay,we know these capabilities exist.
We would like to add these capabilities. Uh, how do we do it to, to our existing products? How do we do it right? Yeah. And how much is it gonna cost? How long is it gonna take? Right?And what are the trade-offs? So, um, so we believe for, uh,Ruby Rail shops, uh, we would like this to be kind of the, the,the default solution. Cool. Um, Santiago has asked, uh,is the idea that Lang Chain RB follows the Python JSS implementation as if thePython JSS implementation is like the standard implementation, uh,in terms of the a p I or features provided, or, um,have you diverged from the current Lang chain implementation, uh, already,or is that gonna happen?So I would say, um, it's inspired by Lang Chain.
Obviously it carries a similar name, um, but it, it,it has diverged a bit. And, and this is, and this is mostly,yeah, and this is, this is mostly just due to the,just due to the ecosystem and differences in the, in the language and,and how we develop applications in in Python versus in real in, in,in Ruby. Cool. Um, I also, uh, so Emily actually has a question, but I'm gonna,I'm gonna ask a follow up question to what you just said first, um,which is,how do you see Link Chain RB fit intothe, the ecosystem of like, working with your LMSs, working with your data?Um,because I think one thing that I see a lot with Link Chain, uh,at least in, in Python, is that, um,it is kind of positioned and framed as this way to chaintogether different ways to interact with your L L M and it doesn't provide asmuch resources to work with your data. Um,so I wanted to get your thoughts on that and how important that would be, uh,to link chain rb.
Yeah, so we're, we're, we're trying to focus on, uh, a lot fewer,um, use cases. So I, I did show agents,uh, but on, on every slide I said it was experimental because it's,it's very difficult to get them to work really, really well. Yeah. Right?So, um,the use cases that I presented were vector search,QA over your data chatbots, and, and that's also the order of our focus, right?So we wanna, we want to make sure that if you're building vector search, uh,within the Ruby Stack, it's, it's, it's an end-to-end experience,right?So you're just pointing to a directory or a data source where yourfiles or your data is, is located, structured, or unstructured,and we take care of rest, right?You just need to make a decision whether you're gonna, uh,pay your money to open AI or, uh, to Google Palm,and whether you're gonna, uh, pay,pay VIS or Zillow or someone else, right? Yeah, yeah, yeah. Uh,so you pick your, you pick your, you pick your l l m,you pick your vector search database, uh, we take care of rest, right? It's,it's, it's plug and play.
Okay. Yeah. That's awesome. Very cool. So it sounds like you're gonna be able to work,you're kind of aiming to work with not just the data, I mean, not just the alls,but also like the data as well,and the agents play is more of like something that's experimental because it'slike not something that is, uh, yet standardized,nor is it something that we know a lot about.
Um,Yeah, it's, um, yeah, I'm not, I'm not sure how many,I'm not sure how many companies actually have, uh, agents,AI agents running in production. Like, yeah, basically the,the research is constantly evolving. I mean, every,every couple days there's a new paper. So I think,I think eventually we'll get there, uh,where AI agents are a lot more reliable. Um,but that's, you know, that's not what we're currently, uh, betting on.
Yeah, for sure. Um,so Emily asked what other tools need to be created in Ruby to fully takeadvantage of LLMs?Uh, that's a good question. Um, there are, uh,there are a lot of, uh,great foundational libraries and Python. Um,I don't, you know, I don't wanna suggest that we, we,we should just recreate them in Ruby. Um, again, even,even when with Link Chain or B, we're, we're putting a,a Ruby spin, uh, uh, on the,on the link chain project, right? It's, it's a Ruby flavored, uh,link chainRubyFlavored.
Okay. Yeah. But I, but I,I think we, we, we will eventually find ourselves that, uh, some, some,some of those foundational libraries, uh, do need to exist in,Um, yeah, that, I think that makes a lot of sense. Uh, Diego has asked,have you contacted Link chain. com team to sync implementations and or make theproject part of the official system, official ecosystem?Uh, we, we do have a link chain or v uh, channel on,on the link chain Discord.
Um, I've had a couple conversations with,with Harrison, but, uh, we've, we've, we, we've never, uh,discussed anything, anything, anything further. Well, do you want to talk about, uh, why or why not you would, um,you know, integrate with, um, the other Lang chain project?Or even if you kind of see this as like a, is this like a, you know,I know you said this is kind of like lane chain with a flavor, but really,you know, someone asked if, if you've kind of spun off from the, uh,the way they've, uh, you know, created Lang Chain, Python, Lang Chain jss, and,um, if this is something that may not even be called Lang Chain, uh,as you, as you grow it. Yeah. Uh, I, um, I,so I don't, I don't, I don't think it's a zero sum game. I don't necessarily think we're, um, if,if you're using Python day-to-day and you love Python, and yeah,you should go with either LAMA index or, or the,the link chain Python implementation.
Um, I do,however, uh, uh,I do however here that long time Ruby developers tryout Python and it just doesn't feel the same. It's, it's close,but it, it doesn't, it doesn't hit the spot. Um,so in that case, I think link chain B is, is, is a great option. Um, cool. Yeah.
Do you think, or I guess this is, this is a, I don't know, um,this is less of a technical question and more of just like a,a curious question is do you think that's, um,that the adoption for Lane Chain RB is gonna be, uh,mainly done by like Ruby, Ruby Shops or hobbyists?Um,I don't see any reason why Ruby Shops wouldn't adopt this tool. SoI'm, I'm curious like what your thoughts are on, on, on like the expansion. Um, well, the hope is definitely that big Ruby Shops would also, um,I don't wanna say back, but, um,the hope is that the big Ruby shops do adopt this tool, right? Yeah. Yeah. Um,obviously it, it, it has to be a good tool, which, which again, uh,circles back to, to the whole feedback loop.
Uh,we don't want to be building, uh, link chain or b for made up,uh, invented use cases, right? Right. It needs to be, it needs to address,um, the small shops, the big shops, and also the, the enterprises using Ruby. Yeah. Yeah, that makes a lot of sense. I mean,link chain Python was adopted through hobbyists,so I was just kind of curious what your thoughts are on that.
I, I think,Yeah, I was just gonna say, I mean, it definitely starts,like any big project starts with A P O C. Yeah. Um,and if we can, uh, and if we can make that as seamless as possible,I think it's a good, uh, it's a good entryway. Yeah. Um,Landon has asked what made you make a link chain implementation versus using,um, PI call, and are there limits to the PI call approach? Also,I'm a huge fan of the good work you're doing.
You got another fan, Andre?Thank you. And thank you. Uh, so hey Halen, thank you for,for joining. Um, actually he's got a great, uh,he's got a great Rails con talk. Oh, cool.
Uh, that, uh,Ruby and a AI and Ruby, um, I, I believe that he gave that, uh,at a Rails conference last year. So it's, it's on YouTube,definitely check it out. It's great to talk. Um, so I,um, I gave Pi call a quick spin. Um,you know, it's still, I'm, I, I'm not, um,um, I, I'm not a fan of those, I, of,of those ideas where one language is, is, is converted into,uh, into a different one.
So, like, I know in the, uh, in the rails world,we had those projects where you can compile Ruby, uh,into client facing JavaScript code, and it just, it sounds like a great idea. 'cause you know, who, like, we all love writing Ruby and, uh,not so much when it comes to JavaScript, um,but it just never worked that well in practice. Um, so that's my,that's my kind of, um, opinion ta take on those types of projects. Um, and again, I think, I think if you have an existing Ruby project,uh, and you want l l m capabilities to our existing project,you're basically trying to decide if we give something likelink chain, RB a try, and we stay, uh,we stay within the same language, right? Like I I, I,I keep stressing the monolith approach. Um,so you drop a jam and you're still in the Ruby environment,or you spin up a separate service,which is a Python container, right? And now, um,and now as an engineering leader, you need to decide, um,do you have capabilities to, uh, to be able to,to add another language into the mix, right? So it's an, it's a separate,it's a separate container.
Um, you need to, um,make sure you have staff, um,that knows Python and knows how to operate that and scale and et cetera. Um, I also, okay, so I have, I, I have a,I have another question outta curiosity. I don't think there's any open questions right now. Okay, cool. Um,so when you were using Ruby Lang chain, uh,is the computation being done service side or the client side?Uh, what kind of computation? Well, you mean like generating,generating embeddings? SoLet's say we're gonna go call like the a p I,we're gonna call an agent and I wanted to like form my SQL query and then likedo a sql, you know, query.
Um, is this all like, uh,is this done? Client side, server side? Like are you calling? So, okay, I,I may not, I just might,I might just not understand how Ruby like monoliths work, but like,the way that I kind of like envision this in my head, right, is like,typically I see a, a difference between front end or like, you know,client side and then back end or like server side. And typically I see everything being implemented on server side and client sidemaking like these a p I calls, right? Mm-hmm. Um, so I just,I was just curious like how Lane Chain RB does that, or if, if,if Ruby on Rails is actually the server side, then then, then, you know,then obviously that makes a lot of sense, but I don't a very comprehensive, uh,grasp of Ruby. Yeah. Um, well, rails is, rails is a web development framework, right?So you can, you, you can build a UI if you want with Rails.
Um,you know, a lot of people opt out to opt in into building a,a React front end, right? And Rails just becomes, becomes an a p i that, uh,your React project, uh, talks with. So, um, LinkedIn,RB currently doesn't have a UI layers, right? So it doesn't,it doesn't have a presentation layer. So all of the demos that I did were, uh,in a console. Uh, so it runs on the backend. Okay,Cool.
It runs on the server. Yeah. Okay. Yeah, I was just curious how that worked. 'cause I was like,'cause as you were explaining, I was like, oh, this is really interesting.
Like,how does that work? Um, so we don't have any open questions. I think we can let this sit for, let's say a minute or so, and then,then we can pass it, uh, back on to, or we can pass back to Emily to,to wrap things up. Um, yeah. In the meantime,I just wanna say like, I think that, uh, I mean, a,I I think that your initiative in creating this is pretty freaking awesome. And,uh, I'm really impressed and I,I like what you're doing and I'm really excited to kind of like stay up to datewith, with how this progresses.
Thank you. Thank you. Um, yeah, it was, it was, it was a pleasure. Um,you know, vector search tends, tends to be a little bit complicated. So even,even just seeing you and hearing you rehash through those concepts, I'm like,okay, back to the fundamentals.
Back to the fundamentals. Um, and,and certainly, uh, thank you to all of the participants that,uh, decided to take some time out of your day and, and tune in. Uh, please do reach out, uh, either to me or you,Jen or Emily for any and all reasons, uh, uh,to, to me via Twitter or LinkedIn or, uh, or create an issue in,in, in the link chain rb, um, repo directly. Thank you,Andre. Eugene, thank you so much for this great session.
I think we'll leave it here today. Um, so thank you. We hope to see you on a future Zillow's webinar and we'll send out, um,some links to, to get involved, um,that Andre mentioned on the call today in the email with the webinar recap. So thank you both for the time in this great session,and we will see you all next time. Thanks.
Meet the Speaker
Join the session for live Q&A with the speaker
Andrei Bondarev
Owner, Source Labs
Andrei has been a software engineering professional for almost 13 years. Among many others he’s been fortunate to make his impact at Spree Commerce (Acquired by First Data), WeddingWire (merged with The Knot), FiscalNote (IPO), National Public Radio, and USA Today. He currently runs a software dev firm, and serves as a Architect/Engineering Manager/Fractional CTO on the client projects. In his free time he enjoys playing tennis and going on long runs while listening to podcasts.