Guardrails in Generative AI Workflows via Orchestration

Abstract: 

With the increase in generative AI language model use and server implementations, there is a growing concern of how models can divulge information or how they can generate inappropriate content. This concern is leading to the development of technologies to “guardrail” user interactions with models, from screening user prompts for inappropriate asks, to checking model outputs to restrict unwanted responses. Examples include but are not limited to models presenting biases and model reveal of personal or proprietary business information.

This talk highlights an open-sourced guardrails orchestrator component which is designed to orchestrate calls and responses to generative model servers and detector servers, helping users easily apply guardrails to their generative AI workflows. We also share our defined taxonomy of detectors that provides a more intuitive organization of capabilities. This component can be easily supported on various AI platforms and is designed to work with popular model runtimes. Users can bring their own implementations of any generation model and detector servers. The orchestrator component can also help guardrail streaming content, enabling a more interactive user experience where users can receive detections along with streaming model-generated content.

We will discuss how we considered several important factors like performance, scalability, extensibility, and maintainability in the architectural design to make the solution production-ready. We will also share examples of use cases enabled, from those with simple lightweight detectors to more sophisticated LLM detectors, and how we adapted for multiple modalities.

Join us in exploring how you can incorporate diverse guardrails in your generative model workflow.

Bio: 

Evaline is a senior engineer working on the watsonx platform engineering team of IBM Research and based in Denver, Colorado. She currently focuses on building guardrails infrastructure for large language model workflows. Her previous experience includes MLOps for IBM's cloud ML offerings.

Open Data Science

 

 

 

Open Data Science
One Broadway
Cambridge, MA 02142
info@odsc.com

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from - Youtube
Vimeo
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google