ML Inference on Edge with ONNX Runtime
ML Inference on Edge with ONNX Runtime


ONNX Runtime is the inference engine for inferencing ML models in different HW environments. Applications of AI are everywhere. This requires ML models that are trained in the cloud to execute on small devices with low power, low compute, and low memory. Such devices are typically used in IoT scenarios. The data captured by these devices are processed before sending the telemetry to the cloud for further actions on the business application. ONNX Runtime has made enhancements to enable execution of ML models in these edge devices to power AI on the edge applications. This session will walk through the workflow to train an image classification model, package in container and deploy to IoT device.

Session Outline
Lesson 1:
Train ML models for IOT applications, e.g. image classification. Start with a pre-trained model to fine tune for specific IOT scenario. Store model in registry. And convert to ONNX.
Lesson 2:
Create the IoT application in Python using the ML model with ONNX Runtime. Package this in a docker image for the target device and register image in container registry.
Lesson 3:
Deploy the container image to the edge device. Run inference sessions and send processed telemetry to cloud for the business application.

Background Knowledge
- Python;
- Machine Learning life-cycle.


Manash Goswami is a product management leader with a passion to solve problems and build products that meet end-user needs. Grown from development - coding, test & verification, systems design & architecture, to product management and business development. Delivered consumer electronics products from smartwatch to tablets. Deep understanding of the Android, Chrome & Windows eco-system. Demonstrated leadership by managing through influence across a matrixed org structure. Capable of driving new initiative/agenda internally or with customers, enable collaboration across functional teams and drive for financial results. Excel at dealing with ambiguity and managing complex, cross-group interactions. Can successfully tie long-term strategic thinking with near-term execution.

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google