Rajiv Shah, PhD

Rajiv Shah, PhD

Machine Learning Engineer at Contextual AI

Rajiv Shah is technical sales leader with a passion and expertise in Practical AI. He focuses on enabling enterprise teams to succeed with AI. Rajiv has worked on GTM teams at leading AI companies, including Hugging Face in open-source AI, Snorkel in data-centric AI, Snowflake in cloud computing, and DataRobot in AutoML. He started his career in data science at State Farm and Caterpillar. Rajiv is a widely recognized speaker on AI, published over 20 research papers, been cited over 1000 times, and received over 20 patents. His recent work in AI covers topics such as sports analytics, deep learning, and interpretability. Rajiv holds a PhD in Communications and a Juris Doctor from the University of Illinois at Urbana Champaign. While earning his degrees, he received a fellowship in Digital Government from the John F. Kennedy School of Government at Harvard University. He is well known on social media with his short videos, @rajistics, that have received over ten million views.

All Sessions by Rajiv Shah, PhD

May 13th 11/13/2026
4:35 pm - 5:35 pm

Hill Climbing: Best Practices for Evaluating LLM Applications

<span class="etn-schedule-location"> <span class="firstfocus">LLMs</span> <span class="secfocus">Intermediate-Advanced</span> </span>

The tutorial covers the most impactful evaluation techniques for LLM applications including the limitations of benchmark evaluation suites, how best to use the model as a judge approach, and finally incorporating human feedback. Additionally, we’ll discuss methods for modularizing complex tasks and employing unit tests to pinpoint issues and drive iterative enhancements. By the end of this session, you should have a practical roadmap for designing and executing robust LLM evaluation pipelines. You will be provided curated Jupyter notebooks, guidance on the latest evaluation research papers, and tangible best practices for evaluation.

May 13th 11/13/2026
4:35 pm - 5:35 pm

Hill Climbing: Best Practices for Evaluating LLM Applications

<span class="etn-schedule-location"> <span class="firstfocus">LLMs</span> <span class="secfocus">Intermediate-Advanced</span> </span>

The tutorial covers the most impactful evaluation techniques for LLM applications including the limitations of benchmark evaluation suites, how best to use the model as a judge approach, and finally incorporating human feedback. Additionally, we’ll discuss methods for modularizing complex tasks and employing unit tests to pinpoint issues and drive iterative enhancements. By the end of this session, you should have a practical roadmap for designing and executing robust LLM evaluation pipelines. You will be provided curated Jupyter notebooks, guidance on the latest evaluation research papers, and tangible best practices for evaluation.

Open Data Science

 

 

 

Open Data Science
One Broadway
Cambridge, MA 02142
info@odsc.com

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from - Youtube
Vimeo
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google