Evaluating Recommendation Algorithms at Delivery Hero


Delivery Hero is one of the leading food delivery platforms that help its customers order food by giving personalized recommendations. The recommendation algorithms used, cater to different use cases: completing the cart, similar product recommendations, complementary product recommendations, etc. The algorithms are powered by advanced data science techniques like gradient boosting, NLP, representation learning to create relevant recommendations. In order to improve user experience we need to constantly improve our algorithms. These improvements could be accurately measured via an online A/B test. Since testing online is always limited by time, the number of users on a platform, and the risk of negatively impacting the user experience which could further impact the business metrics, offline evaluation is used to alleviate this by deciding the best algorithm based on the business objectives.

Key Take Aways:
+ Offline Evaluation
++ Different ways of Offline evaluating recommendation algorithms (depending on the use case and data availability)
++ Usage of traditional evaluation metrics and creating custom metrics based on the recommendation use-case and business objectives
++ Deciding the best recommendation algorithm for the online A/B test

+ Online A/B test
++ Basics of designing an online test
++ Choosing the right statistical test (traditional t-test, bayesian test, bootstrapping, two-part test) based on the type of evaluation metric

By the end of this session, attendees will have a decent understanding of how to conduct offline, online evaluation of recommender systems, enabling them to decide the best algorithm for their use cases.

Background Knowledge:

Python and beginner understanding of recommender systems


Manchit is a Data Science professional with 7 years of experience in the vast field of recommender systems. He has worked in Fashion, and Food domains with industry leaders such as Myntra, and Delivery Hero. Being an applied researcher, he has published papers in esteemed conferences like RecSys, KDD, and ECML-PKDD and has been a reviewer for ECML-PKDD.

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google