The world of Large Language Models (LLMs) is evolving rapidly, and ODSC East is your gateway to mastering it. Whether you’re building your first LLM application or scaling complex pipelines, our expert-led sessions are packed with the insights, tools, and real-world applications you need to become a true LLM engineer.
Explore these cutting-edge talks to supercharge your skills across LLMOps, fine-tuning, RAG, prompt engineering, debugging, and more.
Building the Backbone: Scalable LLMOps & MLOps to Enable AI Integration Across Teams
Pablo Vega-Behar, Director of Machine Learning Engineering at Fitch Group
Learn how The Fitch Group is transforming enterprise AI integration with a scalable LLMOps and MLOps backbone that empowers non-specialist teams to rapidly deploy AI solutions. This session offers senior leaders a blueprint for structuring teams, prioritizing integration requests, and enabling governance for quick, effective implementation. Discover how reusable AI components and streamlined operations can drive business impact without heavy technical lift.
Building and Deploying LLM Application: Jupyter to Production in Streamlit and Beyond
Sudip Shrestha, PhD, Data Science and AI Lead at ASI Government
Get hands-on experience building and deploying an LLM-powered app—from Jupyter Notebook prototyping to a full production deployment using Streamlit, FastAPI, and Docker. This workshop walks you through real-world development workflows using OpenAI or Hugging Face models, with a focus on best practices for performance and scalability. Ideal for developers looking to take their first LLM app from concept to launch.
Achieving Reliable and Responsible Enterprise AI Solutions via Coordinated Monitoring, Evaluation, and Governance
Dr Helen Gu, Founder and CEO of InsightFinder Inc.
As AI systems grow more complex, combining ML models, LLMs, and open-source tools into “Composite AI,” ensuring reliability is mission-critical. This talk dives into the challenges of maintaining trust, accountability, and performance at scale, offering practical monitoring and governance strategies. Real-world case studies will illustrate how enterprises can build responsible, resilient AI systems.
Give Your AI the Keys to the Web
Ariel Shulman, CPO of Bright Data
Discover how to supercharge LLMs with real-time web access, enabling them to navigate sites, extract dynamic content, and perform complex tasks beyond static training data. This session explores the tools and techniques—like remote browsers, search engines, and data pipelines—that make it possible. You’ll leave with actionable insights and open-source resources to bring intelligent automation to life on the live web.
Debugging and Monitoring LLMs in Production
Abi Aryan, Founder of Abide AI
Learn how to monitor and debug LLM pipelines for performance, reliability, and trust at scale. This session covers observability essentials like data drift, hallucination detection, latency tracking, and feedback loops—plus tools like LangSmith, MLflow, Prometheus, and Grafana. Perfect for ML/AI engineers aiming to build robust, production-grade LLM systems across chatbots, RAG, and enterprise apps.
Hybrid Text Classification: Labeling with LLMs and Dense Neural Networks
Mohammad Soltanieh-ha, PhD, Clinical Assistant Professor at Boston University
Cut the cost of text labeling without sacrificing quality by combining premium LLMs with open-source tools. This hands-on workshop guides you through LLM-assisted labeling, vector embedding generation with Sentence Transformers, and neural network training for efficient text classification. Ideal for practitioners seeking scalable, cost-effective workflows using OpenAI, Google Colab, and Keras.
Hill Climbing: Best Practices for Evaluating LLM Applications
Rajiv Shah, PhD, Machine Learning Engineer at Contextual AI
Unlock better results from your LLM apps with a practical guide to evaluation techniques that go beyond standard benchmarks. This session covers model-as-a-judge strategies, human feedback integration, and unit testing for iterative improvement. Attendees will leave with a roadmap, curated notebooks, and proven methods to build robust evaluation pipelines.
Beyond Benchmarks: Evaluating AI Agents, Multimodal Systems, and Generative AI in the Real World
Sinan Ozdemir, AI & LLM Expert | Author | Founder + CTO of LoopGenius
Go beyond academic benchmarks and learn how to evaluate next-gen AI systems—autonomous agents, multimodal models, and RAG workflows—with real-world rigor. This session covers advanced metrics for tool use, collaboration, retrieval accuracy, and hallucination detection, plus trustworthiness and performance monitoring. Walk away with actionable strategies to build reliable, scalable AI evaluation frameworks for production-ready applications.
Adaptive RAG Systems with Knowledge Graphs: Building Reinforcement-Learning-Driven AI Applications
David vonThenen, Senior AI/ML Engineer at DigitalOcean
Learn how to build self-improving RAG systems by combining knowledge graphs with reinforcement learning for smarter, more dynamic AI applications. This hands-on workshop walks you through constructing adaptive retrieval pipelines, implementing feedback loops, and retraining models using open-source graph databases. Perfect for AI developers and data scientists aiming to create responsive, high-performance LLM solutions that evolve over time.
Getting Started with Multi-Agents Applications
Valentina Alto, Technical Architect, AI & App at Microsoft
Dive into the world of AI multi-agent systems and learn how autonomous agents can collaborate to solve complex problems. This hands-on session covers the evolution of AI agents, multi-agent system design, and real-world orchestration using leading frameworks. Attendees will build their own multi-agent app and walk away with practical skills to create scalable, intelligent agent-based solutions.
Entity-Resolved Knowledge Graphs: Taking Your Retrieval-Augmented Generation to the Next Level
Dr. Clair Sullivan, Founder and CEO of Clair Sullivan & Associates, LLC
Enhance your LLM applications by tackling one of RAG’s biggest pitfalls—entity duplication in knowledge graphs. This talk introduces entity-resolved knowledge graphs (ERKGs), showing how advanced entity resolution techniques can dramatically boost both graph data science and LLM accuracy. Attendees will explore real-world examples and gain actionable strategies for implementing ERKGs to power more reliable, high-performing AI systems.
Building Real-World Agentic Applications with LlamaIndex
Laurie Voss, VP of Developer Relations at LlamaIndex
Explore the new frontier of agentic AI with LlamaIndex and learn how to move beyond basic LLM interactions into fully autonomous, goal-driven applications. This session dives into core agent design patterns—like chaining, routing, and orchestration—and shows how to combine them into powerful multi-agent systems. You’ll walk away ready to build real-time, RAG-enabled agents that take structured input and act independently, no chat interface required.
Advancing GraphRAG: Text, Images, and Audio for Multimodal Intelligence
Amy Hodler, Executive Director at GraphGeeks.org
David Hughes, Principal Data & AI Solution Architect at Enterprise Knowledge
Step into the future of GenAI with Multimodal GraphRAG, a groundbreaking framework that blends text, images, and audio into graph-based retrieval systems for richer, more human-like associative intelligence. This session introduces cutting-edge tools like CLIP, LLaVA, and BAML to build workflows that unlock deeper context and explainability from messy, multimodal data. Whether you’re here for hands-on Jupyter demos or just to drop some buzzwords at your next meeting, you’ll walk away ready to rethink what RAG can be.
Ben Needs a Friend: Building a Customized LLM-Based Assistant
Benjamin Batorsky, Lead Data Scientist at GoGuardian
Who needs human interaction when you can build your own AI bestie? This tongue-in-cheek yet technically rich tutorial walks you through creating a personalized AI friend using open-weight LLMs, fine-tuning, RAG, and tool integration. Along the way, you’ll pick up real-world skills in LLM development—and maybe even a new (digital) companion who never flakes on plans.
The 2025 Shift to Smaller Models: Why Specialized AI Will Win
Ivan Lee, CEO of Datasaur
As the hype around giant LLMs cools, the future of GenAI is shifting toward smaller, fine-tuned models tailored for specific use cases. This session explores why specialized SLMs are becoming the dominant force in enterprise AI, offering practical strategies for building efficient, targeted applications. Attendees will gain a forward-looking view of GenAI deployment trends and how to stay ahead of the curve.
LLM & RAG Evaluation Playbook for Production Apps
Paul Iusztin, Senior AI Engineer and Founder of Decoding ML
Building a prototype LLM app is easy—making it production-grade is the real challenge. This hands-on workshop teaches you how to evaluate and monitor agentic RAG systems using LangGraph, prompt management tools like Opik, and LLM-as-judge techniques to detect hallucinations, measure performance, and guide improvements. Learn how to version prompts, analyze embeddings, track metrics, and use user feedback to move from proof-of-concept to polished, reliable GenAI applications.
Introduction to LLM Post-Training
Maxime Labonne, PhD, Head of Post-Training at Liquid AI
Dive into the essentials of post-training large language models, from generating high-quality training data to fine-tuning and preference alignment. This talk covers key techniques, evaluation frameworks, and emerging trends shaping the future of LLM development. Attendees will gain practical insights into tools, workflows, and best practices for enhancing model performance at scale.
Optimizing LLM Inference: Speeding Up AI for Real-World Applications
Zain Hasan, PhD, Senior AI/ML DevRel Engineer at Together AI
Learn how to turbocharge your LLMs with cutting-edge inference optimization techniques that reduce latency without compromising quality. This fast-paced session covers everything from KV caching and attention bottlenecks to quantization and speculative decoding—equipping you with the tools to make your models lightning-fast in production. Whether you’re optimizing for cloud or edge, you’ll walk away ready to squeeze every millisecond out of your AI pipeline.
Basic Theory and Practice of Fine-Tuning LLMs with PPO and GRPO
Seth Weidman , Principal Product Manager at SentiLink
Get hands-on with fine-tuning LLMs using the Hugging Face Transformers ecosystem—datasets, PEFT, and TRL—while diving into the reinforcement learning techniques that power them, like PPO and GRPO. This session blends theory and practice to help you enhance chatbot reasoning capabilities and understand how RL is applied in LLM training. You’ll also pick up practical tips for running these experiments efficiently in Google Colab.
Grading the Graders: Benchmarking an LLM Judge Using Human Grading Patterns
David Mack, PhD, Principal Data Scientist at Humana
How do you measure the effectiveness of LLM agents that generate free-text answers—especially when even humans don’t always agree? This talk shares findings from a real-world study at Humana, analyzing human variability in rating LLM-generated answers and evaluating the performance of an auto-grader LLM Judge. Attendees will learn about inter-rater ambiguity, prompt optimization techniques, and best practices for building trustworthy evaluation frameworks that still keep humans in the loop.
LLMs That Think: Demystifying Reasoning Models
Sara Zanzottera, Senior Developer at BNP Paribas
Next-gen LLMs like OpenAI’s o1/o3 and DeepSeek’s latest models are redefining what we expect from AI reasoning—but what’s really under the hood? This talk demystifies how these models achieve breakthrough performance on reasoning tasks, explores their ideal use cases, and compares them to more general models like GPT-4o or Llama. Walk away with a clear, intuitive grasp of when and why to deploy these advanced reasoning engines.
Methods for Systematically Removing Hallucinations from LLMs
Dr. Sharon Zhou, Co-founder and CEO of Lamini
Tired of hallucinations in your LLM outputs? This hands-on workshop introduces advanced techniques like Memory RAG and Memory Tuning to reduce hallucinations through smarter embeddings and fine-tuning with Mixture of Memory Experts. Learn when to apply prompting, RAG, or finetuning—and how to build mature data and evaluation pipelines that power specialized, reliable mini-agents for larger AI systems.
Building State-of-the-Art, Open-Source Embedding Models
Andriy Mulyar, Founder & CTO of Nomic AI
Go behind the scenes of Nomic Embed, one of the most popular open-source embedding model families on Hugging Face, with over 35M+ downloads. This session unpacks the deep learning breakthroughs—like Matryoshka resizability, quantization, and sparse MoE training—that power its multilingual, multimodal capabilities. Learn how Nomic tackles real-world challenges in scaling retrieval systems and driving innovation from lab to production.
Advanced Techniques for Mastering RAG in Finance
Himani Rallapalli, Senior Applied Scientist at Microsoft
Traditional RAG systems fall short in complex domains like finance—this session shows you how to fix that. Explore advanced techniques like contextual RAG, hybrid search, re-ranking, graph-based retrieval, and retrieval-augmented reasoning to dramatically boost accuracy and relevance. With real-world examples, evaluation metrics, and tools like FAISS, Sentence Transformers, and LlamaIndex, you’ll walk away ready to supercharge your financial RAG pipelines.
From 10 Terabytes to Zero Parameter: The LLM 2.0 Revolution
Vincent Granville, Co-founder & AI Lead of BondingAI.io
Discover how to build a powerful, enterprise-grade RAG/LLM system from scratch—without GPUs or deep neural networks. This session explores how to achieve accuracy, speed, and security while avoiding hallucinations, using techniques like hierarchical chunking, real-time fine-tuning, and intuitive parameterization. Through a case study on NVIDIA’s financial reports, you’ll also learn to design an advanced UI, efficient backend architecture, and customizable agent workflows for truly scalable, multimodal enterprise applications.
Using LLM to Query Data
Rami Krispin, Senior Manager – Data Science and Engineering at Apple
Learn how to turn natural language into SQL queries using LLMs—empowering non-technical users to interact with databases seamlessly. This hands-on workshop walks through system design, prompt engineering, and building a functional Python pipeline that connects LLM-generated SQL to real databases. Based on the open-source lang2sql repo, it’s perfect for anyone looking to bridge the gap between conversational AI and structured data access.
Modular RAG Architectures: Integrating LLMs, Data Stores, and Prompt Engines
Ezequiel Lanza, Open Source AI Evangelist at Intel
Murilo Gustineli, Senior AI Software Solutions Engineer at Intel
As chatbots powered by LLMs grow more sophisticated, building them with consistency and enterprise-readiness becomes critical. This talk presents a microservices-based orchestration framework for Retrieval-Augmented Generation (RAG), showcasing modular components for LLMs, vector stores, and prompt engines. Attendees will walk away with architectural blueprints, performance assessment strategies, and a flexible foundation for scalable, trustworthy GenAI applications.
Conclusion: Your Path to LLM Mastery Starts Here
ODSC East is where LLM engineers are made. With insights from top AI researchers, engineers, and practitioners, these sessions will help you sharpen your technical edge, build scalable systems, and bring transformative AI applications to life.
Don’t just use LLMs—engineer them. Join us at ODSC East and become the LLM expert your team needs.


