Fast Data Access in R and Python with Apache Arrow
Fast Data Access in R and Python with Apache Arrow

Abstract: 

Apache Arrow is a cross-language development platform for in-memory analytics. In this tutorial, I'll show how you can use Arrow in Python and R, both separately and together, to speed up data analysis on datasets that are bigger than memory. We'll cover the fundamentals of Arrow in Python in R, then explore in depth Arrow's Dataset feature, which provides for fast, efficient querying of large, multi-file datasets. Finally, we'll discuss Flight, an Arrow-native client-server framework for transporting data, and show how to set up a server and query against it.

Bio: 

Experienced product manager and engineering leader with a focus on data science and data products, currently working on the Apache Arrow project with the Ursa Labs team. Previously, he leds product development at Crunch.io, where he grew a geographically distributed, cross-functional team from 5 to 25, shaped the product vision and roadmap, and developed processes to deliver a high-quality product tailored to the needs of survey researchers.
He is also an open-source software developer, author of several packages for the R language and contributor to many others in R and Python. Many of his projects sit at the intersection of data science and web services. In his projects and public talks, he is a strong advocate for intuitive user experience/API design and for comprehensive test coverage.
Prior to working in software development, he received a Ph.D in Political Science from UC Berkeley and worked in data science at YouGov, where analyzed survey data and developed tools to streamline data pipelines and workflows. Among the statistical tools and methods he has used professionally and in peer-reviewed publications are experimental and quasi-experimental methods, text classification and sentiment analysis, survey weighting, and web scraping.

Open Data Science

 

 

 

Open Data Science
One Broadway
Cambridge, MA 02142
info@odsc.com

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from Youtube
Vimeo
Consent to display content from Vimeo
Google Maps
Consent to display content from Google