How to run scalable, fault-tolerant RAG with a vector database

Abstract: 

Prototyping retrieval augmented generation (RAG) apps can be easy with modern generative AI tools and frameworks. But the journey from prototyping to production is anything but easy with many traps and pitfalls.

In this workshop, you'll get to see first-hand some of the potential risks, and how the right infrastructure tool can help you to mitigate them.

You will get hands-on with an AI-native, vector databases to provide you with an experiential perspective on how to not only build a proof of concept, but go beyond it. You'll learn about some of the key tools that help RAG-powered applications to achieve scalability, fault tolerance and data isolation.

During the workshop, you will implement features like quantization, multi-tenancy, replication, and horizontal scaling. In turn, you will experience how big a difference each of these can make to the RAG system's performance.

You will be guided through each step, to not only run each piece of code, but also to understand what is happening under-the-hood.

By the end, you'll understand some of the key considerations in selecting your infrastructure tools, and what to consider when going to production.

Background Knowledge:

The participants may choose to follow along, which will require some Python knowledge. The level of Kubernetes & Docker knowledge required is minimal.
However, they can also simply listen & watch, too.

Bio: 

JP Hwang is a developer and educator at Weaviate. He loves to enhance others' learning through everything from hands-on coding projects to engaging and informative talks. He brings a combination of technical expertise, empathy, and commitment, to help ensure that learning is fun and empowering for both the educator and the audience.

Open Data Science

 

 

 

Open Data Science
One Broadway
Cambridge, MA 02142
info@odsc.com

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from - Youtube
Vimeo
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google