The world of Large Language Models (LLMs) is evolving rapidly, and ODSC East is your gateway to mastering it. Whether you’re building your first LLM application or scaling complex pipelines, our expert-led sessions are packed with the insights, tools, and real-world applications you need to become a true LLM engineer.

Explore these cutting-edge talks to supercharge your skills across LLMOps, fine-tuning, RAG, prompt engineering, debugging, and more.

Building the Backbone: Scalable LLMOps & MLOps to Enable AI Integration Across Teams

Pablo Vega-Behar, Director of Machine Learning Engineering at Fitch Group

Learn how The Fitch Group is transforming enterprise AI integration with a scalable LLMOps and MLOps backbone that empowers non-specialist teams to rapidly deploy AI solutions. This session offers senior leaders a blueprint for structuring teams, prioritizing integration requests, and enabling governance for quick, effective implementation. Discover how reusable AI components and streamlined operations can drive business impact without heavy technical lift.

Building and Deploying LLM Application: Jupyter to Production in Streamlit and Beyond

Sudip Shrestha, PhD, Data Science and AI Lead at ASI Government

Get hands-on experience building and deploying an LLM-powered app—from Jupyter Notebook prototyping to a full production deployment using Streamlit, FastAPI, and Docker. This workshop walks you through real-world development workflows using OpenAI or Hugging Face models, with a focus on best practices for performance and scalability. Ideal for developers looking to take their first LLM app from concept to launch.

Achieving Reliable and Responsible Enterprise AI Solutions via Coordinated Monitoring, Evaluation, and Governance

Dr Helen Gu, Founder and CEO of InsightFinder Inc.

As AI systems grow more complex, combining ML models, LLMs, and open-source tools into “Composite AI,” ensuring reliability is mission-critical. This talk dives into the challenges of maintaining trust, accountability, and performance at scale, offering practical monitoring and governance strategies. Real-world case studies will illustrate how enterprises can build responsible, resilient AI systems.

Give Your AI the Keys to the Web

Ariel Shulman, CPO of Bright Data

Discover how to supercharge LLMs with real-time web access, enabling them to navigate sites, extract dynamic content, and perform complex tasks beyond static training data. This session explores the tools and techniques—like remote browsers, search engines, and data pipelines—that make it possible. You’ll leave with actionable insights and open-source resources to bring intelligent automation to life on the live web.

Debugging and Monitoring LLMs in Production

Abi Aryan, Founder of Abide AI

Learn how to monitor and debug LLM pipelines for performance, reliability, and trust at scale. This session covers observability essentials like data drift, hallucination detection, latency tracking, and feedback loops—plus tools like LangSmith, MLflow, Prometheus, and Grafana. Perfect for ML/AI engineers aiming to build robust, production-grade LLM systems across chatbots, RAG, and enterprise apps.

Hybrid Text Classification: Labeling with LLMs and Dense Neural Networks

Mohammad Soltanieh-ha, PhD, Clinical Assistant Professor at Boston University

Cut the cost of text labeling without sacrificing quality by combining premium LLMs with open-source tools. This hands-on workshop guides you through LLM-assisted labeling, vector embedding generation with Sentence Transformers, and neural network training for efficient text classification. Ideal for practitioners seeking scalable, cost-effective workflows using OpenAI, Google Colab, and Keras.

Hill Climbing: Best Practices for Evaluating LLM Applications

Rajiv Shah, PhD, Machine Learning Engineer at Contextual AI

Unlock better results from your LLM apps with a practical guide to evaluation techniques that go beyond standard benchmarks. This session covers model-as-a-judge strategies, human feedback integration, and unit testing for iterative improvement. Attendees will leave with a roadmap, curated notebooks, and proven methods to build robust evaluation pipelines.

Beyond Benchmarks: Evaluating AI Agents, Multimodal Systems, and Generative AI in the Real World

Sinan Ozdemir, AI & LLM Expert | Author | Founder + CTO of LoopGenius

Go beyond academic benchmarks and learn how to evaluate next-gen AI systems—autonomous agents, multimodal models, and RAG workflows—with real-world rigor. This session covers advanced metrics for tool use, collaboration, retrieval accuracy, and hallucination detection, plus trustworthiness and performance monitoring. Walk away with actionable strategies to build reliable, scalable AI evaluation frameworks for production-ready applications.

Adaptive RAG Systems with Knowledge Graphs: Building Reinforcement-Learning-Driven AI Applications

David vonThenen, Senior AI/ML Engineer at DigitalOcean

Learn how to build self-improving RAG systems by combining knowledge graphs with reinforcement learning for smarter, more dynamic AI applications. This hands-on workshop walks you through constructing adaptive retrieval pipelines, implementing feedback loops, and retraining models using open-source graph databases. Perfect for AI developers and data scientists aiming to create responsive, high-performance LLM solutions that evolve over time.

Getting Started with Multi-Agents Applications
Valentina Alto, Technical Architect, AI & App at Microsoft

Dive into the world of AI multi-agent systems and learn how autonomous agents can collaborate to solve complex problems. This hands-on session covers the evolution of AI agents, multi-agent system design, and real-world orchestration using leading frameworks. Attendees will build their own multi-agent app and walk away with practical skills to create scalable, intelligent agent-based solutions.

Entity-Resolved Knowledge Graphs: Taking Your Retrieval-Augmented Generation to the Next Level

Dr. Clair Sullivan, Founder and CEO of Clair Sullivan & Associates, LLC

Enhance your LLM applications by tackling one of RAG’s biggest pitfalls—entity duplication in knowledge graphs. This talk introduces entity-resolved knowledge graphs (ERKGs), showing how advanced entity resolution techniques can dramatically boost both graph data science and LLM accuracy. Attendees will explore real-world examples and gain actionable strategies for implementing ERKGs to power more reliable, high-performing AI systems.

Building Real-World Agentic Applications with LlamaIndex

Laurie Voss, VP of Developer Relations at LlamaIndex

Explore the new frontier of agentic AI with LlamaIndex and learn how to move beyond basic LLM interactions into fully autonomous, goal-driven applications. This session dives into core agent design patterns—like chaining, routing, and orchestration—and shows how to combine them into powerful multi-agent systems. You’ll walk away ready to build real-time, RAG-enabled agents that take structured input and act independently, no chat interface required.

Advancing GraphRAG: Text, Images, and Audio for Multimodal Intelligence

Amy Hodler, Executive Director at GraphGeeks.org

David Hughes, Principal Data & AI Solution Architect at Enterprise Knowledge

Step into the future of GenAI with Multimodal GraphRAG, a groundbreaking framework that blends text, images, and audio into graph-based retrieval systems for richer, more human-like associative intelligence. This session introduces cutting-edge tools like CLIP, LLaVA, and BAML to build workflows that unlock deeper context and explainability from messy, multimodal data. Whether you’re here for hands-on Jupyter demos or just to drop some buzzwords at your next meeting, you’ll walk away ready to rethink what RAG can be.

Ben Needs a Friend: Building a Customized LLM-Based Assistant

Benjamin Batorsky, Lead Data Scientist at GoGuardian

Who needs human interaction when you can build your own AI bestie? This tongue-in-cheek yet technically rich tutorial walks you through creating a personalized AI friend using open-weight LLMs, fine-tuning, RAG, and tool integration. Along the way, you’ll pick up real-world skills in LLM development—and maybe even a new (digital) companion who never flakes on plans.

The 2025 Shift to Smaller Models: Why Specialized AI Will Win

Ivan Lee, CEO of Datasaur

As the hype around giant LLMs cools, the future of GenAI is shifting toward smaller, fine-tuned models tailored for specific use cases. This session explores why specialized SLMs are becoming the dominant force in enterprise AI, offering practical strategies for building efficient, targeted applications. Attendees will gain a forward-looking view of GenAI deployment trends and how to stay ahead of the curve.

LLM & RAG Evaluation Playbook for Production Apps

Paul Iusztin, Senior AI Engineer and Founder of Decoding ML

Building a prototype LLM app is easy—making it production-grade is the real challenge. This hands-on workshop teaches you how to evaluate and monitor agentic RAG systems using LangGraph, prompt management tools like Opik, and LLM-as-judge techniques to detect hallucinations, measure performance, and guide improvements. Learn how to version prompts, analyze embeddings, track metrics, and use user feedback to move from proof-of-concept to polished, reliable GenAI applications.

Introduction to LLM Post-Training

Maxime Labonne, PhD, Head of Post-Training at Liquid AI

Dive into the essentials of post-training large language models, from generating high-quality training data to fine-tuning and preference alignment. This talk covers key techniques, evaluation frameworks, and emerging trends shaping the future of LLM development. Attendees will gain practical insights into tools, workflows, and best practices for enhancing model performance at scale.

Optimizing LLM Inference: Speeding Up AI for Real-World Applications

Zain Hasan, PhD, Senior AI/ML DevRel Engineer at Together AI

Learn how to turbocharge your LLMs with cutting-edge inference optimization techniques that reduce latency without compromising quality. This fast-paced session covers everything from KV caching and attention bottlenecks to quantization and speculative decoding—equipping you with the tools to make your models lightning-fast in production. Whether you’re optimizing for cloud or edge, you’ll walk away ready to squeeze every millisecond out of your AI pipeline.

Basic Theory and Practice of Fine-Tuning LLMs with PPO and GRPO

Seth Weidman , Principal Product Manager at SentiLink

Get hands-on with fine-tuning LLMs using the Hugging Face Transformers ecosystem—datasets, PEFT, and TRL—while diving into the reinforcement learning techniques that power them, like PPO and GRPO. This session blends theory and practice to help you enhance chatbot reasoning capabilities and understand how RL is applied in LLM training. You’ll also pick up practical tips for running these experiments efficiently in Google Colab.

Grading the Graders: Benchmarking an LLM Judge Using Human Grading Patterns

David Mack, PhD, Principal Data Scientist at Humana

How do you measure the effectiveness of LLM agents that generate free-text answers—especially when even humans don’t always agree? This talk shares findings from a real-world study at Humana, analyzing human variability in rating LLM-generated answers and evaluating the performance of an auto-grader LLM Judge. Attendees will learn about inter-rater ambiguity, prompt optimization techniques, and best practices for building trustworthy evaluation frameworks that still keep humans in the loop.

LLMs That Think: Demystifying Reasoning Models

Sara Zanzottera, Senior Developer at BNP Paribas

Next-gen LLMs like OpenAI’s o1/o3 and DeepSeek’s latest models are redefining what we expect from AI reasoning—but what’s really under the hood? This talk demystifies how these models achieve breakthrough performance on reasoning tasks, explores their ideal use cases, and compares them to more general models like GPT-4o or Llama. Walk away with a clear, intuitive grasp of when and why to deploy these advanced reasoning engines.

Methods for Systematically Removing Hallucinations from LLMs

Dr. Sharon Zhou, Co-founder and CEO of Lamini

Tired of hallucinations in your LLM outputs? This hands-on workshop introduces advanced techniques like Memory RAG and Memory Tuning to reduce hallucinations through smarter embeddings and fine-tuning with Mixture of Memory Experts. Learn when to apply prompting, RAG, or finetuning—and how to build mature data and evaluation pipelines that power specialized, reliable mini-agents for larger AI systems.

Building State-of-the-Art, Open-Source Embedding Models

Andriy Mulyar, Founder & CTO of Nomic AI

Go behind the scenes of Nomic Embed, one of the most popular open-source embedding model families on Hugging Face, with over 35M+ downloads. This session unpacks the deep learning breakthroughs—like Matryoshka resizability, quantization, and sparse MoE training—that power its multilingual, multimodal capabilities. Learn how Nomic tackles real-world challenges in scaling retrieval systems and driving innovation from lab to production.

Advanced Techniques for Mastering RAG in Finance

Himani Rallapalli, Senior Applied Scientist at Microsoft

Traditional RAG systems fall short in complex domains like finance—this session shows you how to fix that. Explore advanced techniques like contextual RAG, hybrid search, re-ranking, graph-based retrieval, and retrieval-augmented reasoning to dramatically boost accuracy and relevance. With real-world examples, evaluation metrics, and tools like FAISS, Sentence Transformers, and LlamaIndex, you’ll walk away ready to supercharge your financial RAG pipelines.

From 10 Terabytes to Zero Parameter: The LLM 2.0 Revolution

Vincent Granville, Co-founder & AI Lead of BondingAI.io

Discover how to build a powerful, enterprise-grade RAG/LLM system from scratch—without GPUs or deep neural networks. This session explores how to achieve accuracy, speed, and security while avoiding hallucinations, using techniques like hierarchical chunking, real-time fine-tuning, and intuitive parameterization. Through a case study on NVIDIA’s financial reports, you’ll also learn to design an advanced UI, efficient backend architecture, and customizable agent workflows for truly scalable, multimodal enterprise applications.

Using LLM to Query Data

Rami Krispin, Senior Manager – Data Science and Engineering at Apple

Learn how to turn natural language into SQL queries using LLMs—empowering non-technical users to interact with databases seamlessly. This hands-on workshop walks through system design, prompt engineering, and building a functional Python pipeline that connects LLM-generated SQL to real databases. Based on the open-source lang2sql repo, it’s perfect for anyone looking to bridge the gap between conversational AI and structured data access.

Modular RAG Architectures: Integrating LLMs, Data Stores, and Prompt Engines

Ezequiel Lanza, Open Source AI Evangelist at Intel

Murilo Gustineli, Senior AI Software Solutions Engineer at Intel

As chatbots powered by LLMs grow more sophisticated, building them with consistency and enterprise-readiness becomes critical. This talk presents a microservices-based orchestration framework for Retrieval-Augmented Generation (RAG), showcasing modular components for LLMs, vector stores, and prompt engines. Attendees will walk away with architectural blueprints, performance assessment strategies, and a flexible foundation for scalable, trustworthy GenAI applications.

Conclusion: Your Path to LLM Mastery Starts Here

ODSC East is where LLM engineers are made. With insights from top AI researchers, engineers, and practitioners, these sessions will help you sharpen your technical edge, build scalable systems, and bring transformative AI applications to life.

Don’t just use LLMs—engineer them. Join us at ODSC East and become the LLM expert your team needs.