Abstract: 

The mobile phones and other smart devices in each of our pockets include a sophisticated suite of sensors. Inertial sensors like accelerometers and gyroscopes provide highly accurate measurements of the motion of the device and the user. GPS, magnetometers, and others provide broader context about the location and environment surrounding the device. While many people are familiar with machine learning use cases for imaging and audio applications, techniques for multi-modal sensor data are not as well recognized, nor are they as well developed. Nevertheless, machine learning is a powerful tool for extracting useful features or making decisions with sensor data. Advances in hardware and software, including dedicated chipsets and SDKs, even make it possible to run complex models in real-time on the device. 

In this talk we’ll detail the kinds of sensor data available from mobile phones and other smart devices. We’ll cover capabilities and limitations of the sensors, provide details about the calibration and other pre-processing steps that the mobile platforms typically perform, compare platforms and discuss the range of capabilities, and highlight common issues that may be encountered. We’ll discuss some tips and techniques to consider when building machine learning applications using this data, including applicable neural network architectures (in Keras), feature extraction techniques, etc. Lastly, we’ll review a variety of related machine learning applications, including mobile-phone based vehicle telematics applications in development at Agero, like models for detecting a crash or how someone is using their phone while driving. We’ll also discuss broader applications in fitness, healthcare, augmented reality and beyond.

Bio: 

As a Senior Data Scientist at Agero, Mike spends a lot of time thinking about how to help drivers in need, whether their car has broken down or they’ve been in a crash. He and his team have developed algorithms for detecting crashes and risky driving behaviors using a smartphone, studied crash risk factors using over 2B miles of driving data collected with the MileUp app, modeled demand for tow trucks and when they might arrive late, and much more. Formerly he was an astrophysicist at the Max Planck Institute for Astrophysics in Munich studying cosmic magnetism and developing new algorithms to apply to data from radio telescopes. Mike is a proud father of two, a fan of sci-fi, and enjoys playing games of all varieties.