Building a Scalable REST Endpoint for Machine Learning
Building a Scalable REST Endpoint for Machine Learning


Real-time model serving is a crucial capability to deliver value from data science projects. Unfortunately, many existing REST endpoint implementations cannot scale for large volume and low latency applications. In particular, existing ML platforms with REST serving capabilities can fail in production because the real-time serving infrastructure was not designed to scale while maintaining performance SLAs. In this session, you will learn about: The problems with existing REST endpoints and why they are not production grade. What are the technologies you can use to build out a REST endpoint, what are their pros and cons. How to build a scalable REST Endpoint with Flask, uWSGI, and NGINX. What it looks like to deploy a REST endpoint using this technology stack in the real world. What kind of performance can you expect when using this type of infrastructure?


Lior Amar is the Principal Engineer at ParallelM where he is responsible for MCenter platform. He is an expert with 20 years’ experience in distributed systems development, low-level system programming and HPC cluster management / Linux systems. Before joining ParallelM, Lior was a government researcher working on high-performance computing (HPC). Before that, he was the Founder and CTO of Cluster Logic, a distributed systems consulting company. He has a Ph.D., and Master’s degree in Computer Science focused on distributed systems.

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google