Rajiv Shah, PhD
Machine Learning Engineer at Contextual AI
Rajiv Shah is technical sales leader with a passion and expertise in Practical AI. He focuses on enabling enterprise teams to succeed with AI. Rajiv has worked on GTM teams at leading AI companies, including Hugging Face in open-source AI, Snorkel in data-centric AI, Snowflake in cloud computing, and DataRobot in AutoML. He started his career in data science at State Farm and Caterpillar. Rajiv is a widely recognized speaker on AI, published over 20 research papers, been cited over 1000 times, and received over 20 patents. His recent work in AI covers topics such as sports analytics, deep learning, and interpretability. Rajiv holds a PhD in Communications and a Juris Doctor from the University of Illinois at Urbana Champaign. While earning his degrees, he received a fellowship in Digital Government from the John F. Kennedy School of Government at Harvard University. He is well known on social media with his short videos, @rajistics, that have received over ten million views.
All Sessions by Rajiv Shah, PhD
Hill Climbing: Best Practices for Evaluating LLM Applications
<span class="etn-schedule-location"> <span class="firstfocus">LLMs</span> <span class="secfocus">Intermediate-Advanced</span> </span>The tutorial covers the most impactful evaluation techniques for LLM applications including the limitations of benchmark evaluation suites, how best to use the model as a judge approach, and finally incorporating human feedback. Additionally, we’ll discuss methods for modularizing complex tasks and employing unit tests to pinpoint issues and drive iterative enhancements. By the end of this session, you should have a practical roadmap for designing and executing robust LLM evaluation pipelines. You will be provided curated Jupyter notebooks, guidance on the latest evaluation research papers, and tangible best practices for evaluation.
Hill Climbing: Best Practices for Evaluating LLM Applications
<span class="etn-schedule-location"> <span class="firstfocus">LLMs</span> <span class="secfocus">Intermediate-Advanced</span> </span>The tutorial covers the most impactful evaluation techniques for LLM applications including the limitations of benchmark evaluation suites, how best to use the model as a judge approach, and finally incorporating human feedback. Additionally, we’ll discuss methods for modularizing complex tasks and employing unit tests to pinpoint issues and drive iterative enhancements. By the end of this session, you should have a practical roadmap for designing and executing robust LLM evaluation pipelines. You will be provided curated Jupyter notebooks, guidance on the latest evaluation research papers, and tangible best practices for evaluation.
