Data Pipeline for Retrieval Augmented Generation and Model Training at eBay

Abstract: 

eBay is heavily investing in AI to drive innovation and scale its platform, supporting millions of sellers and buyers worldwide. One focus area is the use of generative AI to increase the velocity of releases and enhance developer productivity. eBay operates on thousands of code repositories across multiple programming languages and possesses a vast archive of documentation on internal frameworks, infrastructure, and features. To boost developer productivity, conversational chatbots/applications and code assistant tools powered by generative AI have been developed. The data fueling these applications are sourced from diverse origins and formats. To manage these data-intensive applications effectively, data pipelines have been engineered to extract data from sources such as GitHub, Wikis, Stack Overflow, and reformat it for use in Retrieval Augmented Generation and the training of eBay's Code LLM models. In this presentation, we will explore the overall architecture of the data pipelines driving these applications and delve into the specifics of data processing within the pipeline. Additionally, we will discuss the methods used for data chunking, the embedding model, and the vector database employed in Retrieval Augmented Generation. We will also cover the data utilized for training eBay's Code LLM, including the various model iterations. To conclude, we will summarize the evaluations conducted for both the RAG and Code LLM models.

Bio: 

Ramesh Periyathambi is a Distinguished Engineer at eBay, driving advancements in developer productivity with Generative AI, including leveraging Retrieval Augmented Generation and fine-tuning Large Language Models to benefit all eBay developers. His expertise in machine learning extends to projects using reinforcement learning, XGBoost, and Logistic Regression, scaled to handle over 1 billion real-time traffic at eBay. Additionally, he specializes in building scalable, data-intensive distributed systems, advanced multi-tier caching solutions, and robust data pipelines. He is an innovator with multiple patents in machine learning, caching, performance enhancement, and e-commerce.

Open Data Science

 

 

 

Open Data Science
One Broadway
Cambridge, MA 02142
info@odsc.com

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from - Youtube
Vimeo
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google