Breaking the ice: How Apache Iceberg is Revolutionizing the Modern Data & AI Stack

Abstract: 

The days of the modern data & AI stack are over. Single vendor solutions to store, process, train and serve analytics and AI are often easier to get started but quickly become expensive and limit your ability to experiment with new tools, models and technologies. Apache Iceberg is an open table format that decouples the storage and data maintenance functions of a warehouse, creating an open, scalable and cost-effective "shared storage" on top of object stores. This shared storage is the basis for the new Lakehouse architecture, enabling a wide range of tools to discover, transform and access data. In this session, you'll learn why creating a shared storage using Iceberg is the future and how it fits into the Lakehouse architecture of your dreams. We’ll go under the hood of Iceberg to teach you how it enables distributed transactions, multiple concurrent writers and how it manages physical data and metadata. We’ll also discuss the various tools supporting Iceberg and how to get started building your new, open, scalable and flexible data stack. I hope you join me on this cool journey, breaking through the ice to find the future of data and AI platforms.

Session Outline:

I’ll go under the hood of Iceberg to teach you how Iceberg enables distributed transactions, multiple concurrent writers and how it manages physical data and metadata. I’ll also discuss the various tools supporting Iceberg and how to get started building your new, open, scalable and flexible data stack. Tools included, Apache Iceberg itself and engines using Iceberg (Spark, Trino, DuckDB, ClickHouse, PyIceberg).

Bio: 

Roy is the VP of Product at Upsolver building a data management solution for Apache Iceberg. Previously, Roy led the product teams for Amazon Athena, AWS Glue and AWS Lake Formation, working closely with small and large companies to architecture and implement scalable data and AI solutions.

Open Data Science

 

 

 

Open Data Science
One Broadway
Cambridge, MA 02142
info@odsc.com

Privacy Settings
We use cookies to enhance your experience while using our website. If you are using our Services via a browser you can restrict, block or remove cookies through your web browser settings. We also use content and scripts from third parties that may use tracking technologies. You can selectively provide your consent below to allow such third party embeds. For complete information about the cookies we use, data we collect and how we process them, please check our Privacy Policy
Youtube
Consent to display content from - Youtube
Vimeo
Consent to display content from - Vimeo
Google Maps
Consent to display content from - Google