General Training Session: Apache Spark for Data Science Part II
General Training Session: Apache Spark for Data Science Part II


Using Apache Spark with Python for Data Science and Machine Learning at Scale Part 2: Assuming attendees been in Part 1 (or have equivalent experience), this hands-on session covers best practices for integrating Apache Spark with all of you favorite Python data science tools, including deep-learning frameworks. We will also learn about using Spark with xgboost, natural language processing integrations (e.g., SpaCy), model parallel tuning with scikit-learn and Spark, and production patterns for inference (i.e., making predictions with trained models).


Adam Breindel consults and teaches widely on Apache Spark, big data engineering, and machine learning. He supports instructional initiatives and teaches as a senior instructor at Databricks, teaches classes on Apache Spark and on deep learning for O'Reilly, and runs a business helping large firms and startups implement data and ML architectures. Adam's 20 years of engineering experience include streaming analytics, machine learning systems, and cluster management schedulers for some of the world's largest banks, along with web, mobile, and embedded device apps for startups. His first full-time job in tech was on a neural-net-based fraud detection system for debit transactions, back in the bad old days when some neural nets were patented (!) and he's much happier living in the age of amazing open-source data and ML tools today

Open Data Science




Open Data Science
One Broadway
Cambridge, MA 02142

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Consent to display content from - Youtube
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google